Difference being that the Feds and Dominion had no formal contact before then. France and Britain had been on and off enemies for centuries and thus one infringing however accidentally on another's territory could easily be misinterpreted as aggression. But finding out some guy you'd never met before had set up a living space in your territory and had no idea it belonged to anyone? I'm sorry but slaughtering them all is just barbaric in that case.
And before anyone says it, I thought the Gorn reaction in "Arena" was also barbaric especially in how they decided to lure in unsuspecting Fed vessels to kill THEM as well. I'm sorry but concerning folks you've never met, the benefit of the doubt is applicable.
Problem is... your applying human values to a wholly alien culture. As far as luring in the
Enterprise, I'd say it was a smart tactic. If they are invaders, it's a good idea to see what exactly you're going to be up against.
How cute

.
The gorn kill everyone - man, woman and child - in a colony, but you shouldn't hold it against them, because they're not human and don't have human moral values.
Newsflash - you are human and you have moral values - or are you willing to throw them in the garbage bin and be the bitch of every alien species that murders your people

?
Any species - the gorn - that has no problem massacring a colony which has no knowledge it is in foreign territory, all that without the gorn bothering to say to anyone they claim that territory as their own, most definitely does not have human moral values - as in, for the gorn, aliens are like insects to be squashed with no remorse. Of course, when it comes to them - you can be sure they'll be really pissed off if you were the one to kill gorn.
For a human, what the gorn did on Cestus III is a glaring war crime.
So - you either throw your values out the window and let the gorn - and every other alien with a problem - massace everyone in the federation they don't like, or it's war.
Or they were perfectly happy to allow New Bajor to exist until the Federation began sending ships/probes deeper and deeper into the Gamma Quadrant from that central location.
At that point New Bajor became a threat to them. And they treated it as such.
Have you even watched DS9?
The Dominion had no interest in peace.
They told no one in the alpha quadrant they exist.
One day, they massacre enentire colony - whose inhabitants had no way of knowing they antagonised the dominion in any way.
They kidnap federation citizens, blow up the ship sent to resque them.
They instigate wars in the alpha quadrant, which kill hundreds of thousands.
They try to blow up the bajoran sun, in ordet to kill BILLIONS!
They start a war of conquest on the alpha/beta quadrants.
They plan to sterilise Earth if and when they win.
But, hey, one should not hold any of these against them, right - after all, they're not human, and that makes it all right to attept genocide/to try to conquer you in a bloody war, etc

.
But - you have no problem being the bitch of genocidal xenophobes, because they're 'not human' and you must not judge them by human standards

.
If someone would gratuitously kill your entire family, you would have no problem with it, as long as the culture of this 'someone' allows murder, yes, BillJ? As for your own values - fuck them!