It's all a dream. The totem was his wife's, not his, so it could follow whatever dream rules his mind set for it. You have buildings closing in on him while being pursued in the 'real' world (classic dream imagery), plus the paranoid pursuit by a shadowy organisation fits within the dream logic of the movie. There are also some inexplicable elements, such as miraculously being allowed through customs within minutes of completing the mission.
It WAS about letting go of his wife (perhaps his subconscioius guilt at accepting his dream state or his real wife trying to pull him out) so he could get back to his 'dream' children, who in the final scene appear exactly as they did earlier in the movie. As soon as he lets go of Mal, her totem keeps spinning - he no longer cares on a subconscious level that it's a dream.
Brilliant ending though - leaving enough doubt for this kind of debate.