Honestly, Voyager really started to go wrong about halfway through season one. Sure the pilot had it's flaws, but they went out of their way to establish the premise. Which was actually an exciting one, and one I really wanted to like. The concept of a divided crew at odds with themselves as much as their environment was an exciting one. Heck that Chakotay guy seemed pretty awesome just in the pilot.
What we got was something fairly different. TNG and DS9 both dedicated a number of their season 7/2 episodes to establishing the Maquis backstory for Voyager. Yet the Maquis really were only a major plot point in a handful of episodes. Heck, they had to alter reality or do holodeck episodes for the Maquis to act like they should've.
What we really got was just a TNG lite with better special effects that they apparently thought compensated for lower quality writing. Not to mention a weaker cast. Janeway was often a self parody and hugely inconsistent. Chakotay was drained of all life after Caretaker. Paris was just an utter Mary Sue. Kim was just laughably pathetic in his antics. Torres was another angry Klingon parody. The EMH really carried the show for the first few seasons but he couldn't do it all. Even then he's not THAT much different from Data. Kes had interesting potential, but by the time she started realizing it... they got rid of her. Neelix? He's just a jerk of the first order. Seven had some potential and some of it was even realized since she actually got a character arc. Still... hard to take her seriously in that catsuit however nice the Borg Spheres may have looked.
The writers in Voyager for the most part did just stick to the familiar excessive use of holodecks and time travel. I'll give credit where it's due and will say Voyager was less episodic and had more plot arcs than TNG, but you could still put a person new to the show halfway through season one, then slip in a season 3 or 4 episode, and have them fail to notice.
They really featured some weak antagonists too. The Kazon? People so dumb you wonder how they could operate spaceships. The whole oppressed slaves overthrowing their masters and taking their technology is interesting... except in execution you could substitute cliche Klingons for them in almost any episode and not even have to change the plot or lines much. The Viidians? A society so screwed up they have to harvest organs... interesting concept, but constantly attacking a heavily armed ship with only 150 people isn't an efficient way to go about that. Wouldn't it be more interesting if Janeway had to try and stop them from harvesting an interesting world or something instead of just another random alien attack of the week? The Hirogen... Klingons on steroids! Again another interesting concept, but just bleh in execution. At least they somewhat didn't contradict themselves even if their "culture" was two dimensional at best.
Overuse of the Q and the Borg both didn't do the franchise any favors. In TNG the Q were a mysterious force out to test the potential of humanity. In Voyager they were a disgruntled sitcom family with a few flashy powers. Watch an episode like Tapesty... they watch Q2 and it's just almost sad. Sure Q's antics and mischievous nature can be fun, but in support of an episode, not the whole reason for it. The Borg two were so overused and so watered down you couldn't take them seriously anymore either. Between the Borg Queen just being freaking insane and stupid, and Voyager "beating off certain annihilation" on a weekly basis, to finally just turning on GODMODE in Endgame.... remember when they used to be scary in TNG? Just another disgruntled villain of the week.
Voyager had the potential for some very interesting and intense drama, unique plot arcs, great character development and change, but just utterly failed to execute. Despite all this, it's not the worst Trek series out there in my opinion... just the most disappointing. It could have been so much more.