I never had a father, either. Though my grandfather was very much a father figure in my life until he passed when I was a high school senior. (It was barely a year after this episode first aired.) But it still completely shatters me every time I watch "THE VISITOR".
Why? Because it's about loss and grief. Everyone either has lost or will lose someone that means a great deal to them. It's a universal thing. Everybody will deal with that pain in different ways, but it's there. And some people simply never fully recover from it. I still miss Pops, almost 30 years later. Same with my grandmother, whom I was especially close to. A week before this Thanksgiving will be 20 years... and there are times when it still hurts.
I can't speak for everyone, but I know I can't be the only one who thinks this... if there were a way to have them come back or I could spend a bit more time with them, I would do it. This is what Jake is doing: giving his younger self a chance to have his father back for more of his life.
And because this is such a univeral thing the episode touches on, and how it was done at the right time of the show (after 3 full years, we got to see just how close this father and son were, so this feels earned) it's why it will always be among the very best of not just the franchise, but in all of television.
Some people just are not able to move on from such a deep loss. It's sad, but it also adds another layer of truth to the episode.