The Breen appeared to be a big thing in Cardassian politics, once we started getting references to them. Nothing explicit, but the animosity certainly was there, as were the territorial disputes.
"Indiscretion" has them as seemingly random baddies, but doesn't preclude deeper animosity; "Return to Grace" only refers to the embassy, and random eps thereafter to their fierceness and untrustworthiness. It's only in "Strange Bedfellows" that we hear of apparently long-running distrust between the Union and the Confederacy, and of Breen territorial ambitions towards Cardassia.
It never gets more explicit than that, but in order to have territorial ambitions, the Breen really need to be neighbors to Cardassia, don't they? So the novels take this and run with it, even if the shows tell us little.
The Breen and the Ferengi are in the "mysterious raiders" category when the TNG era stories begin, even though at least the Ferengi clearly are a known quantity to the Cardassians much earlier on, there being a Ferengi bar on Terok Nor and all. It might make sense to assume that the three cultures are clustered in the same direction, and that Cardassia is the one closest to UFP assets, the conflict with the Union thus blocking the Feds from encountering the other two much. Conversely, it's not automatic that the Breen would be players in the Dominion War if they are behind the back of the Union, rather than smack in the middle of the battlefield. But once they do join, there's little the Alpha Axis can do about it, what with planet Breen being on the wrong side of the battle lines.
Timo Saloniemi