I take a cue from Quantum Leap. Imagine that all of history exists linearly as a very, very long string that starts with the Big Bang and ends with whatever happens after "Utopia". There are knots in this string here and there, that can't be unknotted for whatever reason. This string is all balled up, and exists next to other balls of string that represent other, parallel universes.
Gallifrey is a unique planet outside of all those balls of string.
It's closer to one particular ball than most others, and close enough that most can see it from wherever they are in the ball if they looked hard enough.
Gallifrey is part of the overall fabric, but it, and all the other balls of string, move "forward" in time as whole. A time lord can leave Gallifrey in his TARDIS, visit a point inside a ball of string and travel along it, but when he leaves the ball and goes home, that much time has passed.
At some points in the big ball, a race at that point will become aware of other things happening at other points in the string. Some even learn to leap from one point on the string to another, and in learning the process probably figure out about the other balls of string out there, and that little planet that seems to exist in the constellation of Kasterborous, but that no one can realy get to somehow. Then there's the Daleks, who as a race start to really, really dislike them and jump off their points on the string to knock 'em around a bit.
During the Time War, threads of string in the ball get cut up, frayed, and tied back up. The neighboring balls of string get mixed up in it too. Some parts are lost forever, to be replaced by other bits or new ones entirely, except around the "knots" that are here and there and around which the string always seems to converge back to normal. As a result, whole swaths of history are mucked about with, and entire races disappear or are reborn more than once. This goes on for "years" from the perspective of Gallifrey, looking at the chaos in the big ball and doing their best to keep it from falling apart altogether.
Then the Doctor, armed with "the moment" or whatever, stops it. As a result, Gallifrey is destroyed, the Daleks (who presumably were all gathered around it per the story in "Dalek") are destroyed, and the big ball of string settles down and reforms itself. To top it all off, the entire fabric of all those balls of string is itself tied in one huge knot, and no one or thing from a point before or after that knot can get into it, let alone around it.
So, it's not like it was before, and because all the pieces in the big ball have been strewn around there are plenty of people who don't remember about Gallifrey, the Daleks, or tons of things that just never happened. Other races do however remember how things used to be, and then of course there's that Doctor guy who keeps telling everyone about it.
All that, AND Sam Beckett never returned home.
Mark