The great intelligence said something like...
"It wasn't much compared to the time war, but it was finally too much for the old man"
It was just a bogstandard fight, which he lost.
The spooky shit didn't happen till after the Time Lord was dead and buried.
Rose had enough time energy in her after 5 episodes to bring a Dalek back to full power.
The Doctor was claiming that timespace could fuck up if he was too close to his grave, or learned to much about his death.