Both the producers and the studio felt that the character wasn't working. When the character of Seven of Nine was conceived, there was already a sentiment by the studio that there were too many characters on the show as it was (i.e., they needed to trim the budget) and so someone had to go.
That Kes' character "wasn't working" was an excuse. They were openly changing her in Season 3. She was put in a catsuit before Seven even arrived and they were maturing her character in episodes like "Darkling", "Before and After". The budget reason is true. They didn't want a net addition of characters, so for Seven to appear, somebody had to go. It was originally Kim but was changed to Kes.
*That* should say that the studio/writers felt
Kim wasn't working, not Kes.
It was that 50 most beautiful people of Hollywood, and he was the token Asian on the list. Paramount felt that would look bad to fire him or that having him on was an asset. Digging around, it was People magazine. Influential? Not on regular people, but on Hollywood studios? Maybe. Looks like the issue came out around early May 1997. The Asian-American community really liked it because for them "finally" an Asian male made the list. Here's the list:
#1-10: Winona Ryder, Toni Braxton, Brad Pitt, Claire Danes, Michael Flatley, Roma Downey, Cuba Gooding Jr. The Spice Girls, Oscar de la Hoya, Marie-Chantal
#11-20: Tom Cruise, Mia Hamm, Jared Leto, Juliette Binoche, David Chokachi, Gwen Stefani,
Garret Wang, Lauren Bacall, Tom Ford, Lisa Kudrow
#21-30: Oprah Winfrey (her influence probably got her on that list out of fear on the magazine's part), Rebecca Romjin, Matt Lauer, Leonardo diCaprio, Jennifer Lopez, Andre Braugher, David Baldacci, Kristin Scott Thomas, Harrison Ford, Gillian Anderson
#31-40: Paul Sereno, Jacinda Barrett, Derek Jeter, Michelle Yeoh, Deana Carter, Jeff Gordon, Liv Tyler, Kramer de los Reyes, Lucy Lawless, Jeanine Pirro
#41+: Drew Barrymore, Christopher Cuomo, Gavin Rossdale, Pattie Maes, Jim Carrey, Vivica Fox, where I copied the list from stops numbering and formatting goes wonky from there.
Wang was #17. Look who he was put above: Rebecca Romjin, Leonardo DiCaprio, Harrison Ford, Gillian Anderson, Lucy Lawless, Drew Barrymore, Jim Carrey. He wasn't just
on the list, he was in the top 20. Obviously many would take exception to the list and that ordering of the list (I'd imagine many women in the '90s who read People were pissed Duchovny didn't make the list).
Besides that, Kes was a talented nurse on a terribly undermanned medical staff, and was developing telepathic powers that Troi couldn't even dream of. What did Kim do? Some undefined console work on the bridge, and undefined science/engineering filler. There was no reason to keep him in the show. Oh, but his actor ranked in a magazine article, clearly he was the better choice.
The telepathic flare-up was the excuse to get her off the show. It was created to give Voyager some reason for jettisoning her in space and conveniently to skip over Borg space. Did the writers even think of how they were going to get Voyager to the other side of Borg space before the decision to remove Kes from the show was made? Borg fulfill their end of the agreement? Borg betray Voyager but she gets a transwarp coil in a battle or even from a leftover obliterated Borg ship from the many Species 8472 destroyed? Or the convenient wormhole?.
And Kim was at Ops, basically what Data did on the Enterprise. Ops is sensors, communications, and ships systems. It's a position built into most scripts and usually they want a castmember reading those parts, not a Lt. Ayala. Maybe they could've used the Delaney Sisters. One could start a sentence, the other could finish it.
I think Kes was chosen because she was redundant. Everyone else had a vital role.
* Janeway, Chakotay, Tuvok, The Doctor, Torres: key personnel
* Paris: he was not only the navigator, he was also field medic before Kes. He was also their resident 20th century historian and had a more developed backstory. Besides the disliked by Starfleet & Maquis early on, he was sort of the cynic, the one that balks at hierarchy and regulations, which of course led to his prison sentence, and all this in the context of being a skilled pilot and son of an Admiral who was descended from one of the higher-ups in the Yankees' front office 370 some years 'ago'

. There was more substance in his background than some characters... scratch that, most characters.
* Kim: he was admittedly the most dull. That seemed to be part of the intention, the green ensign, by the book but in no way street wise. Paris made an ideal friend/counterpoint. Of course, while there are many people that aren't wild exciting characters in real life, it doesn't really translate well into dramas or sitcoms. His role in Ops would need to be filled.
* Neelix: people would have loved to get rid of him and his usefulness did end in "Fair Trade", but the Mess Hall was his set and they seem to have wanted a ship cook (need someone to cook for the dorm that is Voyager and feed < 150 people 3 times a day and prepare the snacks for off-hours). And each Star Trek show needed someone with mandatory heavy makeup, Worf was it for TNG, Quark for DS9, Phlox for ENT. Torres' little forehead wasn't enough, so the taxidermists' Frankenstein was made and behold, Neelix!
Based on what transpired, he was obviously the one character they didn't know what to do with. Except for when they decided to growth-accelerate Naomi to an age where the kid can act somewhat and pair Neelix with her, all he got was, what, 2 or 3 lines in the mess hall per episode? Quark was an excellent example of how to use a B-story character or character who only appears in the downtime/interim scenes, not in the action scenes. Notice they stopped even using Neelix as a liaison for first contacts.
* Kes: nurse, she was mainly there for the Ocampan POV (young, curious about the galaxy, but also accelerated lifespan). She came with her own set, hydroponics, which wasn't really too important or utilized in those 3 seasons. She overlapped with Paris as medic/nurse, she overlapped with Torres as love interest for Tom, she overlapped with Seven for blonde in a catsuit. Her character had growth potential, but it was those redundancies that did her in.