Guinan has a giant rifle behind the bar on the Enterprise. And she's a private citizen.
Picard has weapons stashed all over his chateau.
Kevin Uxbridge had a weapon, but it was non functioning. That was a Federation colony.
The colony on Tau Cygni V had armed citizens.
There are lots of examples of Federation citizens having their own weaponry. I think that answers the question.
I gather the most significant thing about every single one of those examples is that the guns never surprised anybody in the slightest.
The heroes never claimed they would be obligated to confiscate the illegal weapons. They never argued the weapons were in the wrong hands. They never as much as said "I'd wish you pointed that thing in some other direction". And as far as we can tell, the heroes are the ones enforcing UFP law and upholding UFP principles - indeed the only party in the entire fictional universe to do so, there being no competing civilian police in evidence.
Two counterexamples stand out in this respect: the Varon T pistol was outlawed because of its specific characteristics, and the presence of Type 8 phasers in Maquis hands was objectionable apparently both because those were guns of Starfleet origin, and because they were heavy ship-mounted weapons where light ones would have been expected. Nobody ever said an UFP spacecraft ought to be completely unarmed, though: the prohibition against arming civilian ships was a dead letter of the UE law in the 2150s only.
Would "specific characteristics" be relevant to the examples above? We never saw any of those weapons fired at city-leveling settings. And Starfleet knows quite well how to wage war on stun (indeed, we see in DSC how this would be far more effective than trying to down Klingons with the kill setting), so limiting the colonial weapons to that setting might serve the frontier needs well enough. Or at least kill would be of no greater help against the invaders that threatened Tau Cygni V or the Uxbridges.
Admittedly, though, Dr. Crater did manage to collapse an old ruin almost on top of Kirk with that pistol of his... Was that heavy stun? We know that stun packs a kinetic punch, such as when the Pakleds throw LaForge across the room with it. Sure, Crater threatens to kill the heroes - but he would manage that with stun all right (see above, or witness him almost killing Kirk with that collapsing rock arch!).
Timo Saloniemi