I've been rewatching VOY on Hulu and it's not as bad as I remember. That being said, I think VOY went wrong when it put the Maquis in Starfleet uniforms at the end of Caretaker. I wish that had taken longer-a season or two, or that some people never got to that point where they felt comfortable putting on a Starfleet uniform. I don't mean the traitors like Seska or Jonas (?), but just some people who were willing to compromise yet not go easily back into the chain of command.
I also think the show erred when it made the Kazon the main heavies early on instead of the Vidiians.