It's pretty damning that other shows can get away with similar arcs without anyone batting an eyelash:
1) In NuBSG no one cares that the Cylons kept showing up even though Galactica's method of FTL put even more distance between them and the Cylons than VOY's Warp Drive did for them and the Kazon.
2) In Farscape no one cares the Peacekeepers kept showing up.
But in VOY, if ANY aliens species showed up more than once of twice there was Hell to pay.
I know what you mean. While it is strange that the Kazon keep appearing, it's not something that's bothering me at all. Why complain if the episodes are entertaining?