I took it to be that one of the recent Doctors (likely Matt Smith) told the first Doctor to start the calculations. He likely then told each of the Doctors to come along the journey in order to make sure there were enough TARDISes to make this thing work. They all forgot because the timelines were out of sync or whatever. In fact, I'm sure the Doctor knew that would be the case.
I kind of wonder if he forgot about Capaldi (but Capaldi didn't forget about him, of course, allowing him to choose to come as well to make sure it worked).