It's hard to take a bunch of bald, hunch-backed trolls as serious enemies.
What does bald and hunch-backed have to do with anything? Do they need hair and straight spines to be taken seriously? Do you not take Picard seriously because he's bald?
As to the larger topic, it's obvious they gave up on them as the big enemy in season one when they decided to pull the Romulans out of mothballs to replace them.
They definitely shot themselves in the foot with the portrayal of the Ferengi. I thought it was cool that they tried to give them alien ways of moving, but the choice made was too bouncy and cartoonish, coupled with some really silly, not thought out things like their love of gold...just replicate a ton of it and send them on their way.