The Doctor remembered Rory after he was deleted because of his own temporal inertia.
Change takes a run at him, but it can't budge him a lick. Kill his grandfather and it might make him yawn. The lad has big brass ones.
Amy had much less inertia, so forgot her fiancée.
Later when they met again in Roman times, Amy still couldn't remember Rory because she had lived a life where time had been rewritten/unwritten and they had never met.
Her memory was jogged (by her engagement ring?) and she amended time behind herself, now having lived two timelines, with and without Rory.
It's all there. Rory's deletion and recreation from Amy's timeline is a dress rehearsal for that happening to the Doctor later on.
Point is that The Doctor just jumped in a crack, and he didn't take the TARDIS with him, but that's how he returned to the "real" world.
Was Brian at the wedding?
River has plenty of temporal inertia herself, which is how she can survive the extraction of the Doctor from Time.
Ditto for Gallifrey and the Daleks.