You would run into the chance of losing a gifted actor who is favored by the audience, and replacing them with a actor who (for one reaon or another) simply doesn't work out.Star Trek series in which people get promoted, killed, and transferred off the show, with new actors and new blood every few seasons could be an interesting change of pace.
Of course the opposite could happen as well, lose a okay character and supersede them with someone the fans simply love.
+