Even as VOY was going through its original run, Neelix was hated, Chakotay and Kim came off as dull and did little more than fill chairs.
What if the PTB had been on the ball, realised that the characters just weren't working and decided at the end of S3 going into S4 they needed to have a massive shake up. How could they have dropped all three at one time? Who would you have wanted to replace them? Would you still want to see Seven strutting around?
How would you have changed things?