IMO this is another issue where canon is self-contradictory and that's just that. I seem to remember the Phil Farrand books discussing the matter and offering contradictory quotes from Kirk in various TOS episodes.
Maybe the best way to see it is that Starfleet are indeed the military, but they're not just
the military. They're NASA, the CDC, the EPA and probably Fish and Wildlife, too.