Like Picard said, the Q are really just glorified flimflam men. Because of their longevity and heighten senses, El-Aurians have the ability to see through the Q's tricks (or "behind the curtain," if you will) and thus the Q's power has little or no effect on them. The Q then feel threatened by this.
Are you suggesting that the Q powers we've seen only actually work on (or even exist for) weak-willed individuals, like some kind of massive Jedi Mind Trick?
That's a bit out there, but a pretty cool concept in a Kantian or even existentialist sort of way.