You really think the 3 million people who stuck with the series to the end were casual viewers?
Two or three common races with a couple random crazy makeup aliens thrown in here and there wouldn't have been more complicated than the things the audience were expected to remember about the Gem'Hadar or about Bajoran politics. And viewers are already expected to remember a few dozen AQ races.
DS9 pretty much dumped the Bajoran politics during the later parts of the show and went for the good vs evil thing, which was a good idea.
Also it's spelled Jem'Hadar NOT Gem'Hadar! Sorry, but your misspelling been bugging me.