Did they know what to do with anyone? Harry Kim's big stand out shows ("Non Sequitor", "Favorite Son", "Emanations") were way, way worse than anything that featured Kes, and he was infinitely blander. I like Tom Paris as much or more than I do Kes, but he was just as wasted in the first few seasons (not that it got much better later on, outside of "Thirty Days" and a handful of others), and outside of being "the pilot" had less of a character. Was he ever believable as "the bad boy" of the crew? Every time Chakotay got the spotlight, I wondered how long it would be before Native American groups started protesting the show.
The Doctor was working. So was Janeway, for the most part. Most of the others were hit or miss in a big way. I don't see how Kes could be singled out as the worst of the bunch. Not even close.
Pretty much why it became "The Janeway-7of9-Dr. Show" starting in the 4th season.