Simple biological needs can explain it for the most part. What's comfortable temperature, humidity, and gravity for a Vulcan may well be deadly to an Andorian, and vice versa. Humans need their sleep patterns set on a roughly 24 hour clock, with reasonably consistent timing, while a Caitan or Tellarite might have two sleep periods a day because they are biologically set up to be active at dawn and dusk. You can have some overlap, but the more species you throw into the mix the worse things get, especially for the long-term. To an extent you can customize quarters and passengers can just deal with things for a few days or weeks, but even with custom quarters for everyone crew scheduling issues will be a major problem. A human might literally go crazy if stuck on a Denobulan ship for too long if he's not allowed to keep his biological clock set properly. Space is not an environment forgiving to sleep deprivation.
The Ent-D, with a mostly-human crew with a few aliens, is probably pretty typical of a Starfleet ship, and I imagine closely-compatible species like humans and Bajorans may see a much greater mixture. I would also expect Starfleet to mandate that everyone serve at a posting where they're not in the majority at least once in their early career.