It's never been established that the Federation isn't colonialist. If anything, since we know they can and do establish colonies, there seems to be evidence to the contrary...
Or so they say.
I don't really believe the Federation is colonialist. I think they are very freedom conscious and respect other cultures. But as long as the dates stand, it looks like the Federation simply claimed a planet that was already settled on. And sometimes some of their behavior is suspiciously colonist at times.
Setting up hidden labs and studying non fed cultures; turning over non fed systems that belong to cultures over to other powers; things like that.
The "wicked" admiral was also acting like a colonialist in certain ways.