Let's look at it through the box office. Ralph Winter believed TFF was so bad it almost killed the movie franchise. But as bad as many say it was, TFF still had the best opening weekend of any Trek movie to that point, and it made some money. On a budget of $30 million, TFF grossed $70 million, worldwide. So it wasn't a box office bomb. Just an unfortunate follow-up story to TVH. And, it justified doing a sixth movie.
INS was not a box office bomb, either. Its opening weekend brought in $8 million less than FC, but it still made $118 million, worldwide on a budget of $70 million. Not a great margin, but not a loss, either. That was only about $2 million less than GEN had brought in, though GEN had almost half the budget. If there was a problem, it was FC brought in considerably more money on a smaller budget. Like TFF, INS was simply a bad follow-up to FC, not really a sign that the franchise losing public interest.
The problem is, NEM was a bomb. Period. NEM barely covered its $60 million budget in worldwide gross. Personally, it's the only Trek movie going back to TMP that I never saw in the theater.
If NEM had at least made some money, any money, there could've been sound justification for another TNG movie for the crew to go out on. Instead, I think Paramount took it as a sign that he TNG franchise had played out its creative juices, fan interest alone could no longer sustain the box office, and it was time to put Trek on the shelf for a while.
Box office and budget information were taken from The Numbers.
http://www.the-numbers.com/movies/series/StarTrek.php