Interesting. After re-reading the panel again and the dialogue I get the impression that maybe Xander is not pleased with his "station" in life. Maybe he's struggling to move on from the events from last season and settle down with Dawn. He really was magnificent last season and I think deep down inside he probably thinks of himself as a Watcher now, or at least a military version of one. He was their General, and now he has nothing really to do. Maybe I'm not expressing my point properly...I think he is in a malaise.
Yeah, he thought he wanted out of the fight and to have a normal life, but now he isn't happy anymore and he'd rather be someone important again. I can see that.
Notice he used the phrase 'It's a wonderful life'. This could be Joss making a reference to the film. Which is about an 'ordinary guy' who gets in a bad situation (because of money problems) and feels like a total failure, and needs to be shown what kind of life his family, friends and townsfolk would have had if he had never been born, to understand that he did make a very positive difference.