Not QUITE that bad, though - aside from seasons one and two, Voyager seemed to “reset” its villains every season, making something of a line with every antagonistic species, marking their space. Especially once Voyager started its various jumps forward.
Well, they did make the mistake of doing another Malon episode after a pretty massive jump that should've gone thousands of parsecs past their territory. And the very existence of the New Talax colony 40,000 light years from Talax is highly problematical, unless they managed to use a Sikarian trajector or the Vaadwaur subspace corridors, neither of which was suggested in the episode.
Revisiting the Hirogen made more sense, though, as they were established as a nomadic people who'd been spreading across the quadrant for centuries.