I mean, the US Intelligence apparatus has like 18 different agencies, sometimes in conflict with each other. There are a wide variety of reasons why Section 31 works for me as both a little more public and a secret agency. In one of the Shatner novels he had an organization that was public in terms of being known, but it was a total black ops organization, who's role was to address threats despite having the most innocuous description of the agency.
What makes Section 31 interesting to me is that we don't always know with them. I like the ambiguity. Do they serve the Federation's interests, their own, or a weird mash up of the two. I find that deeply fascinating because often times there is a drive to protect that leads to an odd sense of priorities.