Two things. One, did you see him in the trailer? I would suggest that clips of footage should give a better picture of what he's like than a staged movie poster.
Two, I think the word boycott was ill-chosen and is the predominant reason there's been a negative response to your comment. If you said you were hesitant to see the movie or skipping the movie because you think it'll be bad, that would be one thing. Boycott carries a connotation of protest and a one-man protest is a bit much over camo pants.