"Enterprise" put an end to Trek as a place for SF and SF ideas. Maybe this was happening gradually over decades, but it reached its last stage here. Enterprise was a mainstream action-adventure show that just happened to take place on a space ship. They aimed for entertainment and conflict like other shows, and gave up on raising questions about technology, scientific discovery, etc... We're not offered new ways of seeing anything... it's more like, did we fill the time? Was there an exciting fight or argument?
----------------------
Despite how much fans like s4 for its continuity fetish, it's all written with dialogue that's just as dull as the other seasons. The talking is there just to get their continuity stuff to happen.
---------------
We see an ancestor of Dr. Soong. At the end, he's in jail, and says to himself-- "Artificial intelligence... maybe that's the way to go..." or something like that. Not clever, groan-worthy actually, but it's continuity with Next Gen, so hooray, for no good reason. Note how utterly uninterested Spiner is in the line, as he says it....
As you said it was happening over decades. But in many ways the core problem was not a change or even a stagnation of the writing. It was that the very technology, science, and in many cases the questions being asked had in fact caught up to the shows. It had drifted from its traditional distant future sci-fi asking phillosophical questions in the abstract to really a more near future setting asking questions about things that are largely already here. Just think how much has changed in those 50 years. How much modern life now resembles those distant future abstracts. It dulls the impact of the stories.
Think of it this way. Jules Vernes stories are fantastic. A great fun read at any time. They hold up well as they are. But you could not write a new one today. It just wouldn't work. The context has changed. The world has changed. Instead of writing speculative phillosophical sci-if you would end up with quaint retro steampunk action adventure.
That's what is happening with Star Trek. It's able to blunt this a bit when it focuses on the characters. But that just swings it more into the action/drama column. And in many ways by sticking to past Trek tropes they have boxed themselves in going forward. I suspect that this is partly why they set Enterprise as a prequel. Because they were quite literally running out of future to go to. Reality was arriving faster than they could create the imaginary future.