I think they only have one Starfleet Academy, and while Starfleet is large, lets not forget that other then in times of war, we not only don't see a lot of deaths, but Starfleet officers may serve for nearly a century. With that, unlike the military, its possible to stay in the same grade (rank) for decades. In the alternate world where Picard never got stabbed in the chest, wasn't he just a junior officer still. He was allowed to advance very slowly over a what, 40 year period? For some Starfleet is a life choice that they do for decades.
Lets not forget we are trying to impose some of our own beliefs on this world. Plus, if we even assume that Starfleet was a thousand ship strong, with an average crew compliment of 500 (which is over estimating) then thats only 500,000 line personnel. Add another 500,000 thousand for support and starbases and we have a million Starfleet personnel. Its really not a lot of people (well, it is, but not as much as some of us like to believe).