^ My guess is the former. Had Marvel wanted Norton back, he'd be back. I mean, for the sake of argument, if Whedon wanted to cast DB Woodside as Nick Fury (borrowing Xander's eyepatch) or Alexis Denisof as Iron Man, do you think Feige would let him?
Come on, don't give the Whedon-haters any more ammo - 'Hey, Norton ain't coming back as Hulk because of that douchebag Whedon! He even said it himself'