They weren't Federation members. They colonised the world before the Federation was founded. So the Ba'ku had first claim to the then unihabiated planet. It doesn't matter that the planet would later fall within Federation space, there are no doubt many inhabiated worlds within the region of space claimed by the Federation that aren't members, can the Federation just swoop onto an inhabiated planet and take what they want?
Maybe, maybe not.
We know that two hundred years prior the Briar Patch belonged to the Klingons. If it belonged to them before the Ba'ku set down on the planet (which nothing in canon says one way or the other) and then that planet was given to the Federation for whatever reason, there would be continuous ownership of it by someone other than the Ba'ku.
BillJ, I think Klingon ownership / transfer of ownership of the briar patch in general, does not change the relevent fact of the Baku's claim or their long history of ownership.
Obviously the Klingons did not kick them out, and may not have even knew they existed, otherwise, their world would look very, very different.
Klingon ownership of the briar patch before the baku arrived gives Klingons/the federation prior claim to the briar patch.
The baku are nothing more than tresspassers.
You actually think tresspassers have any claim to the land they tresspass on?
If yes, the federation doesn't share your opinion. If it would, the romulans would have long ago conquered the federation by settling a few hundred romulans on every non-inhabited federation planet and claiming it as their own.
The Federation's highest laws, and this dates back to Kirk in TOS at least, states that "your world is yours".
The Federation is an alliance for the mutual benefit of member worlds, much more like the United Nations than the Roman Empire.
Except the briar patch is NOT the baku's world - homeworld or otherwise. Tresspassing on a land does NOT make it yours.
And NOT harvesting the immortality particles does a HUGE disservice to the member worlds of the federation and the billions of beings inhabiting them.
Legally, the baku have no legitimate claim on the immortality planet.
Morally, letting billions suffer for the vanity of 600 is obscene.