It has nothing to do with being married, male or otherwise. He's a stalker. Stalkers don't follow oridnary social protocols or know when or how to stop it. He could be potentially dangerous and do something or he could just stick to utterly harmless and creepy. Either way, if he continues, it's time to inform someone like the police, at the very least his wife.
And you might want to consider moving as a step if at all possible should he continue to do this. Or even if he does stop as usually stalkers don't just give up and go home. Literally in this case since he's next door.