I think you're making a distinction where I'm not sure one exists. It's like trying to distinguish between the U.S. Government and the U.S. Military. The latter doesn't exist without the former.
If the Federation were to start to break apart, Starfleet would break apart with it. Just as the military of the U.S. would inevitably break apart if we experienced another war between the states.
It has always been suggested that not all of the species contained under the banner of the Federation are happy about it. Certainly shouldn't be too difficult to push enough buttons to start a collapse.
My point is this: does the audience care enough about the Federation to care if it falls apart? I doubt it. The audience doesn't know
the Federation enough to care.