Question, is Son'a space within Federation space our outside of it? if it's the later the Federation could revoke the Son'a right to travel through Federation space and tell them to leave our territory or we'll destroy your ships.
I don't think Ru'afo and at least some other Son'a gave a damn whether they ultimately had the right to travel inside Federation space. They negotiated because it was, at the time, the path of least resistance. Just as killing Dougherty and forcing the situation ultimately became the path of least resistance.
Whether we assume the Son'a wanted to harvest the rings, or wanted to wipe out the Baku, there's nothing to stop them from sneaking into Federation space, at least that we're presented with. And given how generally inhospitable the Briar Patch was, it doesn't seem likely that the Federation was doing any sort of regular checking-up on the area. Hell, if they were, they surely would have noted the Baku planet's unique properties before it was pointed out to them...unless we want to assume the Federation approached the Son'a rather than they other way around, but I don't think that's likely.
In any event, if the Baku-Son'a dispute is an internal matter, why does the Federation get to tell the Son'a they can't return to the planet they originated from? It's safe to assume the Son'a would be as honest about their motives for wanting to return as they were about being descended from the Baku in the scenario we were presented with.
Hell, they might have even been successful if they'd left it at "We're descended from the people at Planet X, and we want to return home. It's an internal matter and you have no right to stop us."