It's possible that the Dominion got word of the Federation's existence from TOS days ...
Enterprise or some other SF ships could have encountered some advanced alien civilizations that never joined the Feds for example and could have spread 'rumors' about the Feds.
Numerous SF ships went missing during exploratory missions (anomalies), and therefore some could have ended up in Dominion controlled space, or close to where they were.
Further more, for the Dominion to be so self-confident for example and that they would be able to wipe out the Federation ...
I doubt that.
For one thing the Feds were more or less on the same tech-stage like the Dominion (the Dominion having an advantage in shipbuilding and initially with their phase polaron beams that were counteracted), and encountered races such as Cytherians (whose knowledge would take 'decades to analyze' for example).
If the Feds would encounter the Dominion naturally in several hundred years ... the Feds could easily swarm into Dominion territory or simply eradicate their invasion fleets at the borders of where the two territories meet.
Quite simple.
The Dominion and the Founders were arrogant ... not to say the Feds were not in some cases, but still.