The misunderstanding here is that Roddenberry was a purist and with him in full control he would only do "pure" Trek. Roddenberry knew the Hollywood system, the type of people ("the suits") he had to deal with. He knew he had to appear bold and grand to be able to force the necessary compromise inside this system get his vision for the show done as much as possible. Because his concept was and still is unique within the Hollywood system and thus almost not "sellable" to the execs. It is still a wonder that TNG was done the way it was done, as the show goes against anything typical Hollywood. And he also knew he needs to work together with other people and find the best for the job (and that these people are confident & intelligent and you have to stand your ground to find the right compromise in the end to get your vision done). Roddenberry knew he had to set the bar high to in the end get a compromise that does not hurt the artistic vision he had in mind.
I believe, that Trek with TOS, TNG up until ENT were true to Roddenberry's vision and he would have been fine with them in the end. Simply because the guy knew the Hollywood system. I see some of the extreme sounding statements from Roddenberry as part of the deal-making process.
Roddenberry supported Berman (and Piller), because both knew on the intellectual level the value of having Star Trek as a canvas to tell stories about the human condition today in an allegorical way projected into the future. The both knew the value of the so called "Roddenberry Box", defined in the TOS and TNG writers bibles (Piller in his book "Fade In": "The new approach respected Roddenberryʼs rules and by doing so, became a more complex story. He gave his blessing. And I began to learn how Roddenberryʼs Box forced us as writers to come up with new and interesting ways to tell stories instead of falling back into easier, familiar devices.").
He supported Berman as show runner, because pre Trek he was a well traveled documentarian, who knew the world and its problems very well: "Starting a career as an independent film producer, director, and writer, he made documentaries and industrial films. Among his clients were the United Nations and the United States Information Agency. By 1977, he had traveled to over 90 countries making films for these organizations." (Wiki). Berman is a well educated guy, who knows the world inside out (personally), that Star Trek reflects upon critically through allegory.
He supported Piller as head writer, because pre Trek, Piller was a successful journalist, which educated him about the real world, and later censor, which brought him into contact with all kind of information a network broadcasts, and as such, like Berman, knew the real world and its problems inside out: " Piller started out in television working as an Emmy Award-winning journalist for CBS News in New York, WBTV in Charlotte, North Carolina, and WBBM-TV in Chicago, Illinois. However, he then moved to Los Angeles, California and the entertainment side of television in the late 1970s, working as a censor and then a programming executive for CBS. While at the network, he became director of dramas based on fact and program practices." (Wiki).
We got Star Trek true to Roddenberry's vision until 2005 because of Berman and the phenomenon that was TNG because of Piller. Not because both went against Roddenberry, but because both supported Roddenberry's vision for Star Trek as they both knew its value: To teach a common audience through allegorical story telling in a science fiction setting about morals and ethics. Both knew about the problems of the world through their work prior to Star Trek. Both knew the Hollywood system and how to navigate it to create a show that is an entertainment action-adventure show on its surface, but an educational show about morals and ethics underneath.
Roddenberry's optimism was simply grounded on the fact that he knew were mankind was coming from centuries ago and how it developed culturally up until today. He simply extrapolated that cultural evolution into the future (there is a reason why there was a WW3 in Trek's timeline, because events like WW2 showed that often mankind has to look right into the real abyss to get its shit together and try to do better - see post-WW2 United Nations, European Union or Declaration of Human Rights).
If people don't share this optimism, it is not because it cannot be real, it is simply because they lack the imagination to see something beyond what they already know from today (and the discrediting attacks are spawned from the personal interest to not have this lack of imagination be transparent to themselves and others).
In the context of Star Trek, the picture of Utopia on earth was necessary for the allegorical story telling: By having human beings not representing our culture today (in speech and behavior), the writers could project these issues only into alien worlds and cultures (or human settlements on other planets), so that also our problems today could not be depicted in a contemporary way, but had to be put into an alien setting, not using any of our terms (avoiding any contemporary social or political "buzzwords" that would trigger an immediate response in a viewer). So writers for a Trek scripts were forced to always find a new angle, a new perspective on a contemporary issue handled in their story, because they could not use human beings and aliens for a contemporary depiction of this issue (be it a progressive or conservative issue). In that regard, Star Trek shared a core aspect of fantasy worlds like "Lord Of The Rings", as in these also not a single character shares traits with a contemporary individual from today, and a story about a contemporary issue has to always find a new angle, a new perspective by using "alien" characters and cultures.
That's why taking the Utopia aspect away from Star Trek, you leave the 100% allegorical story telling, as human beings can become contemporary now, and as result the Trek story does not provide a new angle, a new perspective, but just repeats what already exists today. Berman and Piller knew of that value and thus protected that aspect as much as possible within the Hollywood system, which hates allegorical story telling (and quite frankly: A lot of viewers hate allegorical story telling too).
Producers hate allegory, because they don't immediately understand the story and what it stands for and if the audience can relate to it and (literally) buy it. Writers hate allegory, because they have to invent new character traits, speech and cultures, instead of just repeating what they personally know or have read in other stories or seen on TV or in movies and cash the paycheck. Viewers hate allegory, because they cannot immediately identify with a character, understand immediately if a story supports their personal social/political viewpoint, or even immediately understand a story and "switch off" their brain to just relax intellectually and let the story wash over them.
So if you want allegory in a Hollywood product, you need "assholes" like Roddenberry, Berman or Piller. Simply because they have to "stand their ground" to get it done. Roddenberry had the name and the brand, Berman and Piller had the knowledge and understanding. All three knew of the value of Roddenberry's artistic vision, of the real world outside of Hollywood, and of the Hollywood system itself.
I think, the Star Trek Roddenberry wanted, is the Star Trek we got under the helm of Berman and Piller, because the compromises, that the Hollywood system demands, were always counted in.