I think he was overwhelmed suddenly having a sort of mouthy child.
I think
If had seen at least some of Alexander while the boy was growing up, it wouldn't have been so bad for him.
I don't think sending him to his relatives was all that bad nor boarding school either.
There are much worse things he might have done.
Not in Star Trek of course but the way people on Earth actually are.
Well, we see Worf indulging the kid a few times, like in A Fistful Of Datas where he plays with him in the Holodeck.