If you refer to my acceptance of such trade-offs in one case and not another, the difference is that here the Federation decided about Federation citizens whereas in the Ba'ku issue the Federation decided the fate of non-Federation citizens.
If there had been a war your government might have the right to redraw the border lines and force you to move. If oil is found under your house your government might also have the right to force you to move and compensate you for it but only your government and not any other government.
Problem is this...
The settlers of Dorvan V left Earth two centuries prior, which would have put it around the beginnings of the Federation. If they left Earth for good it could be seen as them renouncing their citizenship. If they renounced their Earth citizenship prior to the formation of the Federation, their citizenship could be in doubt.
So the Federation may have given away a world that wasn't theirs to give away...