I agree. I can see them losing Hershel, but it would hurt the show to kill off anyone else in the prison group.
Rick -- well.
Carl represents the post-apocalypse generation and there are still too many things to explore from that aspect.
Judith: same reason, as well as the reasons stated above.
Carol and Beth: the characters are finally being developed (particularly Carol) and need more time.
Daryl and Michonne: coolness factor, plus we need to see how they develop after recent events.
Anyone at Woodbury is fair game as far as I'm concerned (with the exception of Tyreese). I wouldn't mind seeing Andrea rejoin the prison group, but it would be equally appropriate at this stage (in a narrative sense) if she died.