***Spoilers from Shows and Books
I think Section 31 often gets too much crap from fans, even though the overwhelming evidence in both from TV and the books show they have done a lot of good work for the Federation.
First I want to address the question of legitimacy. Section 31 while originating in the United Earth Charter was also incorporated into the Federation Charter. From memory-beta: "The Federation Starfleet Charter later incorporated a similar article which, more specifically, established an independent branch of Starfleet Intelligence that held non-specific discretionary power over non-specific matters". So obviously they have legitimacy under the governing document. It is left very vague on purpose so they could have more room to operate.
Second, I want to address the point of unaccountability. In the charter, they pretty much give a nod to unaccountability "non-specific discretionary
power". I think being rogue is a thing. In the name of national security plausible deniability can be a leader's best asset. It allows them not to get their hands dirty but is assured that there is someone doing the dirty work needed to protect the nation's interests. There's probably a chunk of StarFleet Command that knows about 31 but looks the other way due to the fact they understand this. Section 31 also probably has a few members within StarFleet Command.
Third I'll list the positive actions they have undertaken:
In the Enterprise series: Section 31 gives Malcom critical intelligence for the Enterprise to prevent Terra Prime from stopping the formation of United Coalition of Planets. This org would go on to form the Federation.
In DS9: They infect the founders with a virus that will result in death. In any war, decapitating the leadership does significant damage to the organization (see Al-qaeda after the US took out its core leadership...it is dispersed, weak, and unable to project power...and this organization was supposed to be organized in a way where taking out the leadership didn't matter). This is especially true if that leadership is irreplaceable like the founders were. There's some valid argument to the fact that once they were dead, the whole Dominion would collapse since the Vorta and Jem'Hadar had lost their gods and haven't shown the ability to work together without the leadership of the Founders. Regardless, killing the founders would deal a significant blow to the dominion. Yes, it was sad that Oto had to be the carrier, but that's war.
In DS9: They get Koval, the head of the Romulan Tal'Shiar to work for them and get him on the Continuing Committee (basically the board of directors for the Empire). Though it isn't overtly stated, I think this pays off when the Romulanns are dragged into the war. Garek's source is unknown but it isn't a leap to suggest he had sources within the Tal Shiar telling him Vreenak's itinerary. Furthermore, its not a leap also to suggest that with Koval's cooperation the "investigation" into the bombing of Vreenak's ship was a mere formality. It was always going to be laid on The Domionion.
In the books: they aid in bringing to surface the treasonous actions of the Federation President (could have started a war with the Klingons) and permanently dispose of him and his allies.
The one supposed mistake of section 31:
I haven't read Cloak yet but in it they were looking to create a stable Omega molecule when things blew up. I think trying to harness the power of Omega was a good thing. I liken it to the experiements with the atomic bomb. It gave us a source of energy that is much cleaner than coal, and gave us the ultimate weapon that ended the war. Harnassing Omega for energy and as a weapon was a worthwhile endeavor if this is what they were trying to do.
So given all this its obvious to me that Section 31 is a good thing for the Federation. A lot of intellegence agencies' symbol is appropriately the shield (including the CIA). People should remember that ultimately they are there to protect the nation and not give them crap when they sometimes do the dirty but needed work to protect us.