Seasons 4 and 5 also had some very good episodes, where the acting and writing were good. Unfortunatly, the show never came to a level as good as TNG and DS9 were. My girlfriend, who is a big Niner, could watch 3 or 4 episodes of Ds9 after each other, while watching two Voyager episodes after each other is a rarity.
We've discussed this together a lot, and for us (like most fans) the biggest issue is character-credibelity and plot-lines. As she says: Alien of the week gets boring.
The second season had us revisitin the Kazon situation regularly, and gave us a bit of an arc. Things like this were lost after season 3, when Voyager became a true episode by episode series.