Prejudice against the Klingons doesn't really stem from racism at all. It stems from the fact they have a culture that celebrates violence, that has a proven track record of consistent aggression against humanity.
Racism would be irrational hatred based on their appearance or whatever. The hatred of Klingon culture is entirely rational. If it existed, I'd hate it too. Kirk doesn't distrust the Klingons because they look funny. He distrusts them because their entire culture revolves around killing people for fun.
Same here. I was really angry when I saw the first episode of TNG and saw a Klingon on the Enterprise. I was recording it and just stopped the tape and said forget this.
I later came to really like Worf, but it took a while.