It wouldn't surprise me if the core "crew" of the ship was only two or three hundred: the remainder could be made up of a rotating cycle of mission specialists (both starfleet and non-starfleet) and their respective spouses and children.
Oddly enough, in David Gerrold's novelization of "Encounter At Farpoint," he proprosed that the actual operational crew of the Enterprise
-D was no larger than that of the original Enterprise
. The old FASA TNG Officer Manual proposed that ships of the era carried more mission specialists and fewer operational personnel. While both books are non-canon, they do support your idea that much of the ship's actual crew consisted of mission specialists.