First off, milk is calf food... not Human food.
If you aren't breastfeeding as an adult, why the heck would you want to consume bodily fluids of another species intended for their babies (which was btw made by the cow's body for the sole purpose of turning a calf into a fully developed cow)?
Hint: it's because you grew up in a society that tells you its 'healthy' for you and acceptable (it really isn't because it contributes to animal cruelty [artificial insemination/rape of cows so they can be milked, along with their kids taken away from the since birth which are chopped up for food that generates health issues] and many people who consume milk also have issues with their immune system, not to mention, acidic reflux which in turn causes heartburn, and is quite easily linked to other health issues which seem to go away once you stop consuming milk and dairy - also, contrary to popular myths, plants can and do provide more than enough in terms of nutrients).
Is warm milk repulsive?
To me, it always was repulsive, even when I was a kid. Ranging from the smell to the taste.
So, I would imagine that Neelix definitely was up to something.
Plus, what was it that Riker said in early TNG?
Ah yes: 'We no longer enslave animals for food purposes'
And this: 'You've seen something as fresh and tasty as meat but inorganically materialized out of patterns used by our transporters'
So, for all we know, the 'meat' Humans eat in the 24th century could be entirely plant based, but 'designed' from the ground up molecule by molecule using plants as a basis - which btw is something similar being done in real life (without transporters) today through products such as the 'impossible burger' or 'beyond meat'.