However, Neelix was never mean or mistreating in any way, not intentional. He just became very difficult to have around. The same for all his whims which actually did annoy some senior officers from time to time, Tuvok the most.
I remember some comments in one of the books (I can't remember which one right now) when Tom Paris said about Neelix: "Why do his best never turn out to be what we expected from him" which may have some truth in it. In his eager to please, he over-did a lot of things which got the opposite effect. The tragic of his life.
I don't think he became much better in the 4-7 seasons when he was whimpering around Samantha and Naomi Wildman. It was another of his whims, they replaced Kes for him. I guess he had a thing for Samantha as well.
I guess that all of it was the result of his tragic background when he lost his beloved family during the war between Talaxia and Haakonia and therefore left Talaxia to travel among the stars.
But all of that makes Neelix a very interesting character. Really tragic and therefore likeable despite his flaws. The character actually has more depth and dimension that many viewers notice which makes him very interesting-and very likeable!