Neelix is annoying 90% of the time, but when used right ("Jetrel", "Fair Trade"... possibly one or two others I've forgotten, that first one with Naomi Wildman was decent enough) he did actually make for a compelling character. Probably because Ethan Phillips is a competent and likable actor, who unfortunately got too-often saddled with really weak material. And just "lack of substance" weak, like Anthony Montgomery on Enterprise, but "actively makes me want to punch this character in the face" weak, which is harder to recover from.
Still, Harry Kim and Chakotay are more bothersome to me. Neither is written as irritatingly as Neelix tended to be as ensemble characters. But when they got their own episodes, the results were always much harder to sit through than anything focused on Neelix. Is there any Neelix episode even in the same realm of awful as "Favorite Son"? "Non Sequitor"? "The Disease"? "Nightingale"? Yuck, no.
The TOS cast is the best, hands down. I'd put DS9 second, then Enterprise. TNG is a mix of the great and the awful, same as Voyager. I may dislike Deanna Troi more than anyone else, though. Damn nosy Betazoid and her narcolepsy inducing psychobabble.