Hmm, this is a tricky one. On the one hand The Borg are relentless, have no leaders (the Queen is replaceable) and no morale to erode. If they want you assimilated chances are you will be, however, they don't seem to focus on one goal at a time. The collective is spread out so you may be able to withstand an attack of a single cube at a time. Having said that, the more technology you invent the more powerful The Borg will be when the assimilate it.
The Dominion are much more single-minded and are motivated by fear and hate. The Founders can slip into any government and destabilise it, they have the ability to negotiate alliances with other races and engineer soldiers to exploit specific weaknesses. But the soldiers are dependent on ketracel white, if you manage to deprive them of that or if you kill The Founders the Dominion could fall into disarray.
Long term I think the Borg are more dangerous, they will come for you eventually. But in a sustained conflict the Dominion are ruthless, and I don't know the exact figures but I imagine they've caused more Federation casualties than the Borg.