Voyager didnt weaken the Borg, First Contact weakened the Borg. Basically, after that movie the point of the borg became less what they represented, and more what they did. Up until then the main point of the borg was to test Picard and the enterprise crew in kind of a moral situation. Not being able to negotiate (what Picard does best), Riker having to make his decision at the end of BOBW, what to do about Hugh etc. Until that point they were essentially an abstract concept, after this they just became basic run of the mill bad guys. A bunch of aliens under the leadership of one onscreen character who told them what to do, not really any different from other trek enemies.
Making the borg queen weakened the borg as a concept, because it took away the essential element of what they were, and after that they just became something you could tamper with to meet the requirements of the story. This wasnt Voyagers fault, it was First Contacts fault.