Robert Beltran obviously didn't give a damn about his scenes, it showed, and it dragged the whole room down when he said anything. Look at what he says.
"When you're given these throwaway scenes with not much thought put into who's saying it and its importance to the whole story - and I'm talking mostly about seasons five and six here - it just makes me feel like the writers don't care. Therefore the producers don't care, therefore the show isn't that important, therefore why am I busting my ass trying to make this stupid scene worth anything?"
"I just felt let down, ignored and insulted. I think they did a grave disservice to what I thought could have been an interesting character."
What he doesn't realize is that its a feedback cycle. He doesn't think Chakotay has enough to do, so he under performs, so the writers give him less to do. Instead of talking to the writers and maybe pitching his own ideas, he just half-asses it. The result is that whenever he does anything, its a bring down. They should have written him off the show instead of Kes, who clearly brought enthusiasm and energy to her role.