What attracted to her to the untweaked Sullivan was that he fit her genial handyman preference in men. She likes practical manly men, not alpha males but really useful beta ones. I don't think he was that rough to start out with (Fair Haven is a Disney movie) but she's definitely after a romantic build up and for that she needs discussion and mutual appreciation of the arts and/or sciences. That's what she and Kashyk used on each other.
What's really interesting is she changed him to be more "provocative", a Kashyk quality and one completely lacking in Chakotay. Janeway wants a bit of verbal sparring not sappy parables.
If Chakotay figured out what was going on in Fair Haven would he have studied her Sullivan creation and attempted to tailor himself a bit?
Did they give the actor lifts in his shoes for those 3cm? I don't remember him looking taller but I do remember him as not being as tall as Chak.