Neelix was best after Kes was no longer on the series. When they didn't have to play into the romance angle of the character he became a lot more tolerable.
They never really committed to the romance anyway. It's like they planned for it back during writing the series bible, and then realized how creepy it was to have an alien played by a middle aged man dating a character who is canonically only a few years old (and played by someone who is 20). So they basically made them in a "relationship" - but they lived in separate quarters, and Elogium strongly inferred they had never even had sex.
And, as a whole, VOY holds up much better now as fun, interesting comfort food than it did during it's initial run on UPN. I just finished my first full watch of the series (I had only seen about 1/3 of the episodes before) and I enjoyed it a LOT more than how I felt about it back in the late 90's. Even the episodes that were supposed to be "weaker" I found redeeming qualities in.
VOY is a lot easier to tolerate now because it doesn't have the same expectations it had at the time. People kept on hoping it would turn into something else, and were repeatedly let down each week when it...continued to be VOY.