This is in my opinion the main problem of ENT, the bad stuff is obvious and good stuff is often quite subtle. So the series easily turns off many Trek fans (I didn't particularly like it either when I first saw it) and they will never come back.
I think the biggest problem Enterprise
faced during its first two years was that it was dull. I can forgive many, many faults in productions, except that one.