I actually think that Roddenberry had the right idea originally and it was a shame that it wasn't followed through. To me it would make more sense if there wasn't officer/enlisted divide, and it was all just one continuous rank structure. Most of the time Trek does seem to forget that the enlisted exist anyway. To me it would make sense that everyone was an highly trained 'astronaut'; less like navy, more like NASA.
Agreed, it's a genuinely futuristic idea. But no one, GR included, really did anything about exploring it onscreen. When non-officers were considered at all, which was rare, they were apparently about the same as they were in the late 20th century.
In all honesty, when you're dealing with ships with crews of hundreds, you'd need an officer/enlisted divide simply to get the work done. Such a divide does exist in all ocean vessel services today, regardless if they're military or civilian. And since Starfleet is very much in the mold of an ocean vessel service, I don't see why they would do away with that.
This assumes a lot about what would change (or not) in 300+ years. As technology increases, the skill level to use it goes up, too. A hundred years ago a battleship was crewed overwhelmingly by unskilled or semi-skilled labor: about 5% officers, 15% petty officers or skilled craftsmen, and 80% seamen and stokers. There is no way that ratio could work today, the systems are too complex. It took four officers and seven enlisted to crew a B-29 in WW2, ten years later it was five officers and one enlisted on a B-52. Yet a faster-than-light spaceship centuries from now would have a personnel organization basically the same as a seagoing vessel today? Personally, I don't picture that a lot of starship jobs could be filled by a kid straight out of high school, Yeoman Lawton notwithstanding.
I don't think anyone is saying that there wouldn't be levels of lower responsibility who would focus on getting their own job done. Just that it doesn't require division into two castes.
The officers decide what needs to be done, the NCO's ensure it get's done. And a career NCO is an invaluable resource to a new young Ensign or Second Lieutenant - while an Ensign may outrank and CPO, a smart ensign will listen to the advice/guidance of a CPO who may have as much time in the service as the ensign has been alive.
But an outside observer could still reasonably ask: Why does the ensign on his/her first day in uniform outrank the CPO at all? In the US there are 19 grades (not counting warrants which I'd class as parallel officers), why does the ensign start out at level 10 and the chief spend 15 years getting to level 7? Why does the ensign eat dinner with the department heads and "management," while the chief, though she works as closely or more closely with them day-to-day as the ensign, is not welcome? The answer is, when the system was developing, those in authority considered that someone with the right parents could be a leader of men at age 20, and someone from the other side of the tracks never could, even with year after year of skill development and experience.
IMO as the skill level required for every job continues to go up, the harder it will be to justify why Wesley Crusher is the boss of Chief O'Brien right out of the gate. Why not have everyone start out more or less together, and pick the managers/commanders/executives from the entire pool as they prove themselves? If you sign up with a college degree, it gets you maybe two or three rungs up the ladder, but not half way up.