I think it's a matter of semantics and priorities really, per the "Starfleet" article on startrek.com the organisation is:
Branch of the United Federation of Planets assigned to the exploration of space and the resulting scientific, diplomatic and defensive duties. Starfleet began as an Earth agency dedicated to space exploration. To this end, Starfleet utilized interstellar starships and eventually built and maintained a network of starbases. The first warp 5 starship, the Enterprise NX-01, was launched in April 2151 and captained by Jonathan Archer. The ship represented humankind's first foray into long-term space travel. - See more at: http://www.startrek.com/database_article/starfleet#sthash.gtZlfLfT.dpuf
Which suggests to me that the organisation as a whole would best be described as a multi-role government agency but parts of the agency have a collateral military role as the circumstances require it (the UESPA and Andorian Guard divisions per post-ENT Treklit, may have been renamed/merged into the Starfleet Exploratory Division (from various dedication plaques)/Deep Space [Exploration] Corps (from Measure of a Man at some point pre-TNG).
This relationship between the "defense arm" (which is also the main instrument of scientific operations and a major diplomatic force) and the agency as a whole is analogus to the relationship of the USCG (a nominally military organisation mostly tasked to do pretty much anything else) to the Department of Homeland Security.
Branch of the United Federation of Planets assigned to the exploration of space and the resulting scientific, diplomatic and defensive duties. Starfleet began as an Earth agency dedicated to space exploration. To this end, Starfleet utilized interstellar starships and eventually built and maintained a network of starbases. The first warp 5 starship, the Enterprise NX-01, was launched in April 2151 and captained by Jonathan Archer. The ship represented humankind's first foray into long-term space travel. - See more at: http://www.startrek.com/database_article/starfleet#sthash.gtZlfLfT.dpuf
Which suggests to me that the organisation as a whole would best be described as a multi-role government agency but parts of the agency have a collateral military role as the circumstances require it (the UESPA and Andorian Guard divisions per post-ENT Treklit, may have been renamed/merged into the Starfleet Exploratory Division (from various dedication plaques)/Deep Space [Exploration] Corps (from Measure of a Man at some point pre-TNG).
This relationship between the "defense arm" (which is also the main instrument of scientific operations and a major diplomatic force) and the agency as a whole is analogus to the relationship of the USCG (a nominally military organisation mostly tasked to do pretty much anything else) to the Department of Homeland Security.