One bad unprofitable film doesn't kill a franchise. If it did, the James Bond series would have ended in 1974. Or Star Trek V would have been the final Trek movie. It wasn't Insurrection or Nemesis that caused the TNH films to fail, it was overall franchise fatigue. There was already too much bland Trek on TV - for free. Why would people rush out and pay
to see more Star Trek when the various TV series aren't thrilling people? Nemesis bombed before anyone saw it - there was zero interest in the film during its opening weekend, not even fans came out (except me, I was there opening night in a pretty empty theater). Even Star Trek V had a big debut weekend; Nemesis had nothing. It was second to Maid in Manhattan, a crappy J-Lo rom-com. This had nothing to do with Star Wars or Lord of the Rings, people just didn't care. "I'll wait for the DVD" seemed to be the general feeling.
Insurrection wasn't a huge success, but people would still say "the last movie wasn't great, but maybe this one will be better" if they retained in interest in the series. The blandness of Voyager and Enterprise, regardless of their otherwise relative merits, got people tired of Star Trek (not enough of the public paid attention to DS9 for it to get any blame).
What got people interested in 2009 was the previews, which promised a thrilling, action packed, crowd pleasing film with young Kirk and Spock. No prior commitment to the franchise was required. Clean slate. People were excited over Trek, proving the concept is still appealing to people, but the style got stale. Star Trek was like a loaf of bread. Hot and fresh and delicious that, over time, got old, stale and moldy. It was time to bake a new loaf.