I've been thinking about this. We never really get a good sense of how many inhabited worlds there are in the galaxy of Trek. We see the Borg designate a new species number 8472, so assuming there is nothing more to their system than a linear count, they encountered 8471 before that. Assuming their system only counts intelligent life (or it would be in the trillions) that gives us some sense of the scale of intelligent life in the area the Borg have explored, which is pretty big and includes areas of Delta, Beta and Alpha quadrants at least. Maybe a guess of approx 9000 intelligent species per quadrant is reasonable?
It may be that there is a natural upper limit to how big the Federation could reasonably grow, assuming it did not absorb any of the other big empires of the TNG era. 350 member worlds does seem small, about 4% of the guess above in the alpha quadrant, but maybe we simply don't know enough about the geopolitical forcings of the 25th through 30th centuries to know what stopped the expansion. As you could say "why does the US peak at 50 states? That's barely 6% of the land mass!" and completely miss the reasons it stopped expanding rapidly.
We DO know however that the Borg encountered over 10 000 species by late 24th century (Voyager)... and assimilated a good portion of those species (but likely not all of them - such as species that didn't offer anything of interest to the Borg, or because they couldn't be assimilated - like 8472)... in some cases when they go assimilating species, they assimilate them in totality (aka, no refugees or anyone left with intact individuality).
An interesting note: We do know they encountered 8472 in Voyager's third year of travel through the DQ... or most likely (shortly before Voyager stumbled upon Borg and 8372).
And only 2 years later, the Borg encountered (and assimilated) species 10026.
Thanks to TW technology, it seems the Borg encountered 1554 more species in just 2 years.
The Federation being limited to slower warp would of course progress slower... but also, this doesn't strike me as convincing because the Federation had numerous options for far faster Warp travel and alternate means of propulsion (especially after Voyager returned)... so, I doubt speed would have been a limiting factor, and we could say that it had mainly a lot to do with Federation entry requirements and we know from 'The Voyager Conspiracy' the following:
CHAKOTAY: That's quite a theory. Perhaps you haven't heard of the Jankata Accord?
SEVEN: No species shall enter another quadrant for the purpose of territorial expansion.
CHAKOTAY: The Federation signed it, the Cardassians signed it, and Captain Janeway would be the last person to violate it.
Its possible that this is what mainly limited Federation expansion further into the galaxy... but 691 years (after the 24th century - 691 years from 2378 would give you the year 3069... when 'The Burn' may have happened) is a VERY long time for those accords to still be in place.