It wasn't the Baku's world, to begin with, and they were occupying only a minuscule portion of the planet. It would be like me claiming possession of Japan!!! At the very least in order to claim possession of the planet, they should at least occupy a significant portion of its surface. For all, we know there could be people on the other side of the world that the Baku have never met!! Anyway, even if they owned the entire planet that wouldn't give them ownership of everything above, which is a little absurd as they could claim the entire universe that way. Plus the particles are hardly in orbit of the planet, as I understand it they occupy the entire solar system, which clearly belongs to the Federation. Of course, the ST writers have generally no idea of how space distances work so they sometimes make the stupidest of assumptions.