the jealousy over Kes thing really hurt the character early in the series run. They took it way too far. Also, Neelix' extroverted personality could go overboard to the point of being creepy. People don't like some guy forcing himself ito your conversations or trying to "cheer you up" when you just want to be left alone. He was also annoying with the "Mr. Vulcan" stuff toward Tuvok.
Still, the problem wasn't with Ethan Phillips, and when he was given good material "fair trade" and "mortal coil" he did well. The character got better in later seasons, and ironically was the only character to get a satisfying conclusion to his storyline.