So, I've been watching Voyager through for the first time since it aired, and I'm up to season 5. Here's my thoughts season per season.
1. This is cool, back when Voyager was a little more committed to the 'lost alone in hostile tribe space' aspect.
2. Seska's calculating schemes and the serialized stuff with Vidians is pretty interesting, I don't know why they didn't do more serial stuff later. Started off kinda weak but had a strong finish.
3. ...Meh. Can't wait for Seven, this is boring.
4. Hirogen are boring. Year Of Hell is good but not as good as I remember, I need a new 'Favorite episode'. All they did this year was have Seven complain a lot.
5. Wow. Did they hire an entire new writing staff? This is way better than every other season so far.
I vaguely remember enjoying season 6 for the most part and enjoying season 7 less. So I would say, Voyager was better with Seven, for reasons that have nothing to do with Seven.