I'll quote one of the Family of Blood members directly on how Dr. Who punished him and his family after they went on a murderous rampage across time to try to steal the Doctor's immortality.I don't remember that ep, so what happened in that episode?
Baines: He never raised his voice. That was the worst thing. The fury of the Time Lord. And then we discovered why. Why this Doctor, who had fought with gods and demons, why he'd run away from us and hidden. He was being kind.
(Clark, wearing heavy chains, falls into a pit.)
Baines: He wrapped my father in unbreakable chains, forged in the heart of a dwarf star. He tricked my mother into the event horizon of a collapsing galaxy to be imprisoned there forever.
(Jenny is sucked out of the Tardis in deep space.)
Baines: He still visits my little sister once a year every year. I wonder if one day he might forgive her, but there she is. Can you see? He trapped her inside a mirror, every mirror. If ever you look at your reflection and see something move behind you, just for a second, that's her. That's always her. As for me, I was suspended in time. And the Doctor put me to work standing over the fields of England, as their protector.
(Baines has become a scarecrow.)
Baines: We wanted to live forever, so the Doctor made sure that we did.
For what it's worth, this scene is still very controversial among Dr. Who's fandom as some still believe this ventures way into disproportionate retribution/revenge for someone who's supposed to be the hero of the show.