Let's say that it's maybe 1000 or 2000 light years in area during Kirk's era (2260s) for the UFP. How many ships would be needed to patrol such an area?
According to some simple web searches, at distance up to 100 light-years, there would be about 12,930 stars. Kirk told Cochrane that they were on a thousand worlds. Making some assumptions like the % of M-Class planets (or terraformed into M-Class planets) are found verses non-habitable solar systems, we can get some idea of volume of space the UPF encompasses. If life can be supported on 10% of these solar systems (and this seems low based on my TOS observations), then the UFP could be smaller than envisioned fitting inside 100 lightyears in radius. Put it at 0.5%, then we need about 200 lightyears radius. Again, possibly smaller than thought.
Of course, the UFP could be irregular in shape making it bigger in some of its linear dimensions. For example, in
Squire of Gothos, it was suggested they were about 900 light years from Earth
and they were on their way to Colony Beta Six, so, the UFP has colonies out about 1000 light years. Based on this piece of evidence, 1000 light years from Earth seems plausible for UFP space.
I see an exploration zone around the UFP space to be another few hundred lightyears for new exploration. Room to grow.
How many ships? Easy, 12 Starships during TOS. We also know the Federation has a fleet of smaller exploration/cargo/survey vessels like the
Antares and
SS Beagle. If we assume the ratio at 50 to 1 of smaller vessels to Starships, and we get 612 ships. With a high degree of mobility, I can see one Starship per 100 planets. Probably less since the safer inner systems don't need that much protection plus strong member worlds contribute to their own defense. Both these thoughts will tend to push the Starships out to outer "surface" of the UFP bubble. Also, how many times did we see it taking months or years (or decades) for a Starship to investigate something, further suggesting a smaller fleet. YMMV

.
With an utopian Earth, most people don't want to risk their lives. It takes a rare person defend the UFP with their life and to venture out to explore the galaxy. I like the feel of a smaller fleet where each ship and even each crewman holds a large degree of importance. They are the best of the best. UFP also needs staff for all the starbase and station installations where the vast numbers of Federation personnel are found.
Starfleet Academy graduates officers. A few hundred per year. On the Enterprise, the oldest few officers are in their forties with some in the thirties and most in their twenties. It is a job for the young. The attrition rate for Starships must be quite high. I guess they either leave Starfleet or transfer to ground stations.