I'm sure plenty of people were seduced by the twinkle in Tom's eye or go weak at the knees even for Capaldi.
For what it's worth, my stepmother thinks Capaldi is clearly the best looking Doctor.
And, according to Tom Baker's autobiography, being the Doctor was a great way to get laid. Kinky women liked putting on the scarf.