Man broke time in the Age of Ultron. Things don't work how they are supposed to anymore... Or Hank tripped some rule that no one was ever aware of before.
The Watchers disgust, naybe Hank is not letting them go back. Is it possible that he's lying? He's tied them to a tent pole and not told anyone?