Yeah, I always assumed that Damar was posted on the freighter and that's where he met Dukat. The loyalty came later, maybe when Dukat decided to fight the Klingons and Damar subscribed to this idea of what a "true Cardassian" is.
In a Cardassian library
or in a Cardassian gallery
"Reagan, it appears, is really only an ardent unionist if the unions in question are in Poland" - Stephen King, Skeleton Crew