Well, it was inevitable in VOY's case since because otherwise they'd all be dead in 15 seconds. And in case you'll say "then use another enemy", they tried that repeatedly and in every case it failed due to negative audience reaction (NOT because of the writing).
Then the writers collectively failed on two counts - creating 'new' enemies or antagonists and preserving an 'existing' one as the danger they had been established to be.
Perhaps the mistake was in trying to have 'enemies' in the first place instead of making more out of the journey of discovery through a massive uncharted region of space.
There was nothing wrong with the enemies VOY created, it was the negative audience reaction that got rid of all of them (whether that reaction was justified or not). The audience was just pre-disposed to giving VOY's creations a harder time than the other shows.
Basically, this would've been like if DS9 had stopped mentioning the Dominion after "Jem'Hadar" because there was a bad reaction to them (and if the episode was written EXACTLY the same).
So it was a chain reaction: VOY has to stop using their original creations, meaning they have to keep using the Borg, which causes inevitable villain decay.
If the audience had just accepted the original aliens VOY made, then it would be a different story and they'd have more than the Borg to fall back on.