That's like saying that because you're a commoner who's never been to Africa, you don't know what a giraffe or zebra looks like.
No, it's exactly like saying someone who (in his own words) spent centuries doing nothing but breaking rocks, knowing little of the other worlds jumped at the chance to experience them and fell in love with the first one he visited. After which it'd be impossible for him to learn of anything going on in the other realms, since he's stuck down here with is mortals.
Which, again, is like saying someone who hasn't experienced life in Africa has no idea what a giraffe or zebra looks like. Even miners know what they look like. They may never seen one in person, but saying they're utterly ignorant of anything other than rocks is just an incredibly
stupid thing to say.
The guy was also knowledgeable enough to be a fucking
professor, knew all about an ancient and powerful Asgardian staff as well as their lore, and so on and so forth.
Your theory (sorry, theories actually have facts to back them up; your utterly random guess) just doesn't hold up at all, and there's
nothing that has even hinted at it on the show.