I agree. I've never worried about him dying or something. That's absurd. I also don't like the assumption that if he died the worst part about it would be the fact that the series won't be finished as opposed to how sad it would be that a person, y'know, died.
Very noble of you... given that there are people dying on this planet every second do you cry for all of them too?
It certainly sucks when people die but I can't bring myself to wallow in pity for all of them because dying is the inevitable sequel to living.
I don't know GRRM, all I know is his work. So yes, from my point of view the biggest impact on me would be his work being unfinished.
And FYI, his mortality is on his mind as well, he's given the show runners broad stroke info on how the story plays out in case he were to die.