I will add a final thought as to why Voyager disappoints me personally: this show didn't just have a good premise, it had an awesome premise, easily the best of all the Trek shows.
A Starfleet crew and a band of Maquis rebels are cast to the far side of the galaxy and must make their way home through countless light years of uncharted space, including the hostile territory of the Federation's most deadly enemy. Sign me up! I can't emphasize enough how rife with potential such a premise actually is. One of the best of any sci-fi show I can think of off the top of my head.
Actually, I had issues with VOY's very premise myself. It needed work.
For starters, the Maquis. They just were not the source of conflict that the show seemed to utterly require (I don't even think that the show needed it, to be fair). They were never developed properly as being anti-Fed enough when VOY started to serve as a real opposite group.
Now, if the other crew had been ROMULANS, then it's another story.
Also, this "lost from home" stuff. It's just Gilligan's Island in space because you KNOW all the "shortcuts" they'd find won't really do anything to get them home and they'll likely find a bunch.
Being always on the movie means they'll never stick around long enough to develop their own original aliens enough to make them on par with the Klingons, Romulans, Cardassians etc. DS9 had the stationary advantage meaning they could keep bringing folks back to develop them better.
So right off the bat, you have shallow aliens who'll never stick around long enough to be developed to the level of past Trek aliens, you have two groups of people who don't really have much reason to oppose one another and several reasons to work together, and traveling around space in some manner that will never be beneficial.
Yeah, real great premise.
And the Borg, I'm sorry but the Borg have no real place in VOY except to get trashed. You need lots of cannon fodder for the Borg to really work in a story, at least fodder to be killed off to keep the Borg threatening without harming the main characters or the ship. You know why the Dominion never lost their threatening name in DS9? Because DS9 always had tons of cannon fodder to sacrifice to the Dominion to keep up their threat. VOY didn't have anything to keep up the Borg's menace with. And the only choice they had when they encountered the Borg would be to destroy them because you can't outrun the Borg and the Borg can track you anywhere.
And since VOY wasn't going to be allowed to build up some fleet of cannon fodder ships to take on the Borg in their way (since it would cost too much and require serialization), the only choice was to just keep having the Borg lose without harming anyone (since UPN wouldn't let them just stop using the Borg).
So no, VOY did not have THAT great a premise. It was pretty limiting actually. Possibly the MOST limited of all Trek shows.