Honestly, what probably turns me off the most is probably the monster-of-the-week aspect to the first season. While I said I really liked X-Files, I should maybe clarify that I loved the mythology episodes, the MOTW could be fun enough due to the interesting cast of characters, but generally, that type of story that just showcases weird, wacky pseudo-science bores the crap out of me.
Curiously, I'm the opposite. I loved the 'monster-of-the-week' stories on X-Files, but didn't care for the mythology. With 'Fringe,' I find myself liking the mythology more than the MOTW stories.