The guys in black outfits on DS9 are obviously a ground force of some sort, even though they all use naval style ranks. That is the interpretation the novel continuity has taken, "Colonel" is just a nickname used for Admiral West.
And yet most episodes have them visiting planets (ie ground), and not all the exploration can be done in a few hours, so the idea of 'medium term' units (perhaps as noted, similar to Green Berets or Marine Raiders) doesn't seem that far out (the blue shirt security from WNMHGB and the navy shirted HRT from TFF could be from these units) that can serve as regular security but are also trained for extra duties as needed.
If a character had at some point specifically stated that Marines didn't exist, that would be proof. As things stand, it's unproven one way or the other. Up until TVH the federation president wasn't mentioned, but no one ever said prior that such a person or office didn't exist. Countries with nuclear weapons still have rifle squads. The US Navy has masters-at-arms, but they're not Marines.
Nor are they SEALs (which the AR-558 and Adijion Prime units are plausibly equivalent to), in fact MAA aren't/weren't even source rating for SEALs except during a brief period in the 2000s.
I saw no indication that the personnel on AR558 were anything like special forces, more that they were infantry. In the case of Ajilon Prime, again not special forces, they were (imo) just a random collection of Starfleet personnel from different specialties.
I agree with your analysis. If there were a "more military branch" of Starfleet or in service to the Federation, we would surely have seen them in DS9 during the Dominion War. As it stands, DS9 has multiple seasons that provide the best "proof" we have is that Starfleet is a multi-role organization that includes (not limited to) defense and military. Keep it simple folks!
GENE'S VISION! TNG WRITER'S GUIDE EXCERPT! PICARD'S LINE IN PEAK PERFORMANCE! ADMIRAL FORREST IN Ah you know what, fuck it. I just don't care on this subject anymore.
They were just infantry though, they had engineers and at least originally medical and logistics personnel (though they appeared to have mostly died before the Defiant arrived). Also, as with the MACOs, the relatively high rank of the unit's CO (Starfleet Captain-by-title, either Captain or Commander by rank*) given the small size suggests it's a specialist unit rather than general service infantry unit (although something along the lines of the USCG's DOG, the RAF Regiment or a SEALs/Seebees combo might be a better description). * Whereas an infantry company would only rate a Lieutenant.
A company is usually commanded by a captain, or possibly a major. A lieutenant will command a platoon. A corpsman would be expected in a company, the "logistics personnel" (who I don't recall) could have been part of the company headquarters unit. The engineers might have been attached to the company to investigate the Dominion communications device.
Just going to throw this out there for everyone: TOS Redshirts are an extremely bad example for proving the Federation doesn't need specialized planet-based troops. Screw Section 31. I'm glad David Mack killed it off in the Litverse. As for some kind of "Marine" force? Starfleet's entire canon attitude and structure is utterly ill-suited to planetary engagements, expeditionary or otherwise. Even having such a force under Starfleet at all would be a mistake, I think. It would be a separate service--a direct successor to MACO in mission, if not in name. The evidence doesn't point to said planet-based service being canon, but then there are a lot of canon things in Trek that are completely unrealistic and bug the hell people who have even an inkling of how these things actually work. Call it "wanky" to point out those flaws and try to find solutions that endeavor to stay in the overall spirit of Trek if you want, but that's just the way it is.
Starfleet uses naval ranks and Siege of AR558 confirms this is still the case for ground combat personnel so a company commander would be a Navy Lieutenant (O3), whereas the investigation unit and it's security team were led by a Starfleet Captain (or at least a lieutenant commander with the title of Captain)* and an additional full or lieutenant commander as XO. * NB: Although it is indirect evidence at best, it's interesting to note that the MACOs also seem to have uprated their ranks similarly, as they have a Major commanding a Platoon (which as you note above is usually a Lieutenant's job), which would suggest that "Captain" Loomis was actually a full Captain. Presumably Commander Parker would have led the technical team, whereas Loomis (and Lt Parkin) led the troops.
Section Thirty-one, cut off one head and two grow back. Yeah, problem with all that is the original size of the unit was about 150 personnel, typical size for a small company, and it was originally commanded by a Captain who was referred to by that rank (and who was killed prior to Sisko's arrival), not a Lieutenant. I thought the red shirt who was escorting Captain Decker gave a pretty good accounting of himself, rare exception. I think that was a case of that particular unit being sent on such a special high profile mission, that a higher ranked officer was assigned to command that MACO unit. In my own head .... I doubt a MACO Major (O-4) typically commands that few people.