I think you might like a site called Historypin
. One of its features is very similar to what you suggest, though not exactly an elaborate 3D environment. If you go to the map you can see where people have added historical photos. Some of the photos have a little Google Street View symbol on them (the orange little man), and if you click on it then you can see the current and historic photos overlayed. There's even a bar along the bottom you can drag left and right to increase or decrease the overlay. Very interesting stuff! It looks like there are quite a few in San Francisco.