Is it known why the sphere design was rejected, or was it simply not the right shape for the era when saucers were seen as the peak of space traveling?
In-universe? In the TNG era, a sphere-hulled design was portrayed as a hospital ship. Perhaps there has always been a demand for sphere hulls, but in "peaceful" applications like that, and the sphere-hulled
Daedalus or whatever (we have never seen that class name actually associated with the sphere-hulled tabletop model!) was a "peaceful" ship from the mid-22nd century while the saucer vessels from that era were "warlike" or "exploratory".
One would then be tempted to think that saucers go faster. But that's explicitly not true in the TNG era, where both the hospital ship and the modernized E-D travel at warp 13 when in relative hurry. "Peaceful" might instead have something to do with shielding, the saucers being easier to protect somehow.
Or then it's the obvious thing: a saucer has a smaller silhouette in the standard encounter between two starships, which is nose-to-nose. A sphere presents a maximal silhouette to the enemy from all directions; a clever captain can minimize the silhouette of a saucer by maneuvering right, and it takes even less effort to minimize the silhouette of one of 'em Vulcan spindles. So spheres are preferred when the assumption is that nobody is firing at you much (and the other attributes of a sphere, such as possible ease of shielding or ease of construction or whatever, thus dominate), and saucers when you need to dodge death rays and hide from targeting systems.
Out-universe? I gather the sphere looked sorta passive and sluggish, there being no bow and no "forward impetus" to the shape, so Jeffries was told to do something sleeker. He kept the attractive simplicity of the circle shape by using a saucer, even at the risk of drawing something looking like a trite old UFO, and Roddenberry approved. Of course, bits and pieces of the story actually survive - folks like
Shaw or
Harvey etc. here would have the dirt. But the full story probably wouldn't have been told, unless it was very specifically asked when Jeffries was interviewed.
Headcanon? I go with the novels that say the sphere-hulled little
Daedalus was a mass-produceable wartime desperation measure, akin to the small escorts the RN built out of whaler blueprints in WWII. Which means I think the sphere was simply easier to build than the saucer, while in fact being an inferior shape in combat (for the silhouette reason and others), and quickly disappeared from military use after the Romulan War concluded and desperation evaporated.
Timo Saloniemi