Enterprise really started to get interesting with season 4, when Brannon and Braga were replaced as head writers by Manny Coto. But, by then it was too late, audiences had already had enough.
Let's be fair, though. While Star Trek fans might have loved the fourth season for its continuity and fan service, it was not as if those elements make good television of themselves.
The fourth season was pandering to those reduced viewing figures, not remedying them. I don't think it was "too late", because that implies that it could have worked if it arrived earlier.
I suspect that had the show tried that in its first two years it might have been better-loved by fans, but would have lost viewers just as quickly. If not quicker.
And that's the problem with a chunk of the fandom, imho that they have this extremely narrow view of what kind of stories the franchise is "allowed" to tell, what characters "can" or "should" be used and what constitutes "real" Star Trek.
This in turn causes the franchise to be stale and in love with its own past.
I agree entirely. The next Star Trek showrunner should not look to Star Trek fandom for advice. I want a showrunner who doesn't give two shrugs about "the base", realising that the "the base" is a couple of hundred extremely vocal fans on the internet, and has nothing to do with actually providing what is necessary for a television franchise to succeed. Which is a much larger number of people who really like the show, rather than a core who will never be satisfied until a few years after you go off the air and they can feel nostalgic.
Look at Doctor Who, there are extremely vocal and vicious elements of fandom who have always objected to every major change in the franchise. (There are those who refuse to acknowledge the new series, those who refuse to accept that Moffat is different than Davies.)
What kept that franchise alive (and made it thrive) was the realisation that you need to grow the base, rather than pander to existing fans. Compare the insular approach of the wilderness years to the "come one, come all" attitude of Davies and Moffat.
(Which is a large part of the backlash to both - "Davies turned the show into a soap opera!" or "Davies has a gay agenda!" were two of the most frequent criticisms of the Davies era, while "Moffat is telling stories that are too complicated!" or "Moffat is moving away from the characterisation of the Davies era!" are the two more popular mainstream criticisms of Moffat, ignoring the somewhat questionable tumblr crowd.)
I think a version of Star Trek that is going to survive has to basically treat us as "nice to haves" rather than "essentials." It needs to throw out any and all expectations, refuse to get bogged down in continuity, and be utterly unafraid of doing its own thing.