From TOS "Tomorrow is Yesterday"
CHRISTOPHER: Must have taken quite a lot to build a ship like this.
KIRK: There are only twelve like it in the fleet.
CHRISTOPHER: I see. Did the Navy
KIRK: We're a combined service, Captain. Our authority is the United Earth Space Probe Agency.
CHRISTOPHER: United Earth?
KIRK: This is very difficult to explain. We're from your future. A time warp placed us here. It was an accident.
CHRISTOPHER: You seem to have a lot of them. However, I can't deny the fact that you're here. With this ship.
This implies to me that Starfleet grew out of the Navy (and other military branches).