In the five Doctors they kept calling Doc #5 "the kid" or "the young one" or things to that effect, and they treated Doc #1 as the wise old man, instead of the arrogant youth he likely would have been seen as (imagine you are 72, and and meet yourself at ages 54, age 36 and age 18. You'd probably find your 18 year old self an insufferable youngster, and your 18 year old self would probably think he's more clever than the lot of you combined.)
Given how young the actor who played Doc# 5 was, it would have been really funny if the other Doctors all called him an old geezer, and treated his young body as signs that he's having a sad little midlife crisis.
On the other hand, when First Doctor first showed up in the Fifth's TARDIS and started tramping around like he owned the place, Five mentioned to Tegan as an aside that he's mellowed in his old age.
Did he? Sheesh, it has been too long since I've watched a couple of these episodes. That is a cool line.