The kazon can't be the only race that would diminish the Borgs perfection.
Negative perfection.
The more perfect they become the higher their standards, the higher the bar is raised, the less species they will accept into their collective until every unassimilated species (of which there must be thousands and thousands) in the galaxy is too pathetic to align with the Borg.