I don't think it's too bad. Say he's an alien who travels through space and time with human companions. You'll miss a few details (the Rory-Amy subplot), but the episode otherwise makes sense. I watched it with my brother, who has only passing familiarity and he was fine.
River Song requires some time, but you can wait until she appears before you do that.