Problem is, the Borg are fundamentally boring as a recurring evil. They work better as the occasional threat seen every few seasons or one big story in a series and that's it. Maybe if they had the Borg be a plot device driving a conflict and had the characters deal with the situation the Borg had created without really encountering the Borg it would work, but as a real confrontation VOY HAD to weaken the Borg otherwise they'd be destroyed in 10 seconds and the show over.