It always struck me as odd that a lot of Star Trek fans would see a story starting in a – relatively – dark place and ending up on a hopeful note reaffirming the positive values as a dark story overall. I've seen a lot of people express the view that Star Trek's optimism is rooted in the fact that the positive outcome has already happened in the backstory, and any kind of threat to that positive outcome inevitably robs the setting of its optimism. And considering how some people went ballistic over the Federation being shown as weakened due to a galactic catastrophe in Discovery's far future, it also feels like some Fukuyama-esque "end of history" thinking is at work here, which is quite understandable due to the geopolitical environment TNG was made in. Unfortunately, this means that the utopian future of the Federation is often not only treated as the prerequisite backstory of the setting but also as its inevitable endgame – with utopia having already been achieved, all problems have irrevocably been solved forever and there's nothing more left but to expand our horizons, never to worry about what's back at home. Because of this, any kind of examination of how this utopia can work, and also any kind of threat being shown to it, whether external or internal, will be seen as attacks on the very fabric of Star Trek itself, even if they are practically always averted by the actions of the heroes at the end.
That being said, the Federation not being shown as a utopia is nothing new to Star Trek; TOS definitely wasn't one and despite Picard's speeches in TNG about how humanity has left every kind of societal ill behind permanently, the TNG era shows themselves didn't shy away from showing it wasn't exactly a flawless and perfect place either. Of course, it was quite easy to disregard that fact when the stories always focused on Starfleet crews, and even when they went to peripheries where people still struggled, they still had their safe havens in the form of their starships (and, by extension, Starfleet itself behind them) that shielded them from most of the adversity that defined life for the people of that frontier. This series simply follows a group of people who don't have that luxury. None of this means the future hasn't always been portrayed as an optimistic one, however.
The main difference, in my view, between the TNG era and the current one is that unlike TNG, modern Trek consistently argues that utopia cannot be taken for granted. We can't just sit back, declare all problems to have been permanently solved and expect everything to stay perfect forever. Just like how our own political apathy has led to a world of extreme political polarization that snuffs out any meaningful attempt at reform as well as the constant encroachment of authoritarianism worldwide, Picard shows a Federation wounded by all the wars of the last twenty years which has led to a general mindset of averting risks and focusing inward, preserving what they still have left instead of risking they lose it by overextending their hand again. Utopia has wavered because the people haven't been working to preserve it, and now Picard has to remind the Federation and its people what they are truly capable of. However, at the start of the series, he himself is like a Fisher King whose own emotional state reflects a Federation that has similarly given up. He has to find his own resolve and faith again before he can help the Federation do the same. At the beginning, he goes to Clancy with a mindset of "I'm the only one willing and capable of doing it against a Federation that doesn't listen", with predictable results, and by the end he's recovered enough to be willing to sacrifice his own life to remind the androids and the Federation that our values are always worth fighting for. He's back to his old self and the world finally listens to him.
Discovery and Picard are all about not taking values for granted and losing ground because of apathy or complacence.
