They didn't want the Kazon because they offer nothing to advance the collective. At best they would simply be extra drones, not worth the time and effort. The Kazon were not technologically advanced, intelligent, there was nothing special about them.
I presume you must have something to interest the Borg to warrant assimilation, it's likely they have assimilated Kazon in the past and found them to be worthless.
Think of it this way, people are driven by money usually, but if you saw a one dollar coin in a very difficult to reach area, you probably would just leave it, if it was a fifty though, you'd be trying to get that.
Kazons are 1 cent coins.