I don't think it's right to say they despised humans, so much as they would do any nasty thing they needed to if it made them profit. But despite their angular mincing gait and ridiculous accents, other than Damon Bok it wasn't personal.
I like the way the writers called out the hypocrisy of the Ferengi portrayal. Quark goes around preaching "What's worse, a laissez-faire economy, or slavery and the holocaust?"
It is true, in early TNG Ferengi warships were a military match for Federation ships, and later in TNG then in DS9 their ships were barely armed and defenseless.