But I kind of liked the notion that Starfleet didn't have a sophisticated ground military.
I don't. Any organization that *doesn't* have a military will soon be dead. The Federation must have one. The fact that it's lasted this long is proof of that.
Now what this organization is actually CALLED, is anybody's guess. Could be the Federation Marine Corps, Starfleet Marine Corps, the MACOs (who knows, they may still exist), or whatever, but there must always be a form of ground troops. Indeed, in several episodes of DS9, Federation ground troops (that specific phrase was used - 'ground troops' ) were mentioned. So we know they exist.
You can't take Starfleet shipboard security officers, for example, and put them on the front lines. They are there to enforce security on board ships and starbases. Ground fighting is completely different. You can't have one do the other. That would be like taking a security guard at Wal-Mart and sending him to Afghanistan. There must always be specialization.