There are the usual reasons already mentioned above.
I'll add a new one--
Before it got out of the gate, ENT faced a LOT of preconceptions from fans.
Being a prequel series, fans already had their own ideas about what a pre-TOS series should be like. This goes for the design of the ship, to the history of the future, to the technology depicted, to the types of characters, to the look of the sets, everything.
Every fan can imagine what it was like between our time and Kirk's era. When ENT showed up, right from the start, it was gonna piss off people with something
they didn't get right (according to any given fan).
The people are too contemporary, the ship is wrong, the sets don't look right, the ship is too advanced, that's not how Star Fleet was started, this is wrong, that's not right, whatever.
Fans were already gonna have their own ideas about that 22nd century. ENT was never gonna please everybody about everything.
I see that as a unique problem for ENT that other ST series didn't face, where they were going forward and fans didn't "know" any better about what "should" be.
I guess some fans got pissed at the start about something ENT got wrong, and never stayed aboard.