Honestly, my impression is the Voth are just aloof --- they feel as though the Borg are so below them they aren't worth their time. We've seen they actively seem to discriminate against mammals in general and what are the borg primarily (as seen on screen at least)?
As for why the Borg hasn't assimilated a lot of the races we've seen in the DQ, in another thread regarding the episode, "Living Witness" (S04E23), it was discussed why the Federation still hasn't reached the planet where The Doctor was some 900 years later and someone did a great calculation that proved it simply takes a very, very, very, very long time to explore, catalouge, make contact, open relationships and move on; and that's assuming a constant rate of expansion. If its not constant, or you got in a few wars along the way, it would set you back a few hundred years.
The Borg are, no doubt, in the same boat.
We know the only reason the Borg proactively scouted out the Alpha Quadrant because of the events of Star Trek Enterprise's Episode "Regeneration" (S02E24). If you want to go into Path to 2409, you can track it back to even more specifically to the events of the Columbia in "Gods of the Night"; we learned the origin of the Borg were because of a random mutation with the Caeliar species and a random strain of the Omega molecule.
Though I'm not going to live in a fantasy world and claim TPTB had that much planned out, from a story standpoint, we could assert the ONLY REASON the Borg Ship was around the Federation to begin with was to:
A.) Seek the distress signal sent by the Borg from Star Trek: First Contact
B.) To discover more information about their own origins.
C.) To discover more information about the Omega Molecule.
Otherwise, there's likely no reason the Borg would have been in the Alpha Quadrant.