I liked Neelix to a point... he certainly had some great roles in some of the episodes, playing an important and useful part of the Voyager crew (remember, with the replicators off-line, he was the only one capable of soup kitchen volume cooking). And his "connections" during Voyager's early trek through the Delta quadrant were critical at various points. Otherwise, I don't know if it was the writing or the actor, but I often found him a bit annoying/obnoxious and in some episodes painfully so.
Phlox had his share of annoying/obnoxious moments, but fewer than Neelix. Definitely.
I do wonder if much of Ethan Phillips' personality came through in Neelix, though. In the times I've seen him since, I can't help but see Neelix without the makeup.