A few months ago I took some family and friends on a trip to San Francisco (we live a little over an hour away) where we participated in an Amazing Race type of scavenger hunt around the city, called Urban Adventure Quest
. It's mostly on foot and you can do it with any group of people, at any time, you just log on to the site and answer the questions, which lead you to lots of interesting sights and locations. I especially enjoyed going through Chinatown and to the Cable Car Museum, and of course ending at the pier at sunset. It was a really fun adventure. For anyone visiting SF for the first time, or even those who have lived nearby all their lives, I would recommend it. (By the way, we are the Scranton Stranglers on the leaderboard. We were in first place back when we completed the race!)
Of course, since it will only be you and your spouse, it won't necessarily be the same experience as having a group of people.