In the earlier seasons I don't think it was as bad; they actually go back to a lot of things and address them. It's the later seasons that make me cringe due to the continuity.
Two things that always baffle me:
1. In "Dark Frontier" Chakotay tells Janeway that she fiddles with her combadge every time she's about to drop a bombshell. She
never does this at any other time, before or after this episode.
2. At the end of "Unimatrix Zero" we see Janeway in sickbay after being de-borgified, and she seems perfectly fine. It's all "oh just these damn spinal clamps" and nothing more.

Picard was haunted for years after his assimilation and yet Janeway's all fine and dandy?!
That is what bothers me about Voyager's continuity; they address something once (mostly in the later seasons) and then never go back to it.