I always figured Spock was helping him to forget the PAIN she caused, not Rayna herself.
"Damnit, Bones, you're a doctor. You know that pain and guilt can't be taken away with a wave of a magic wand. They're things we carry with us—the things that make us who we are. If we lost them, we lose ourselves. I don't want my pain taken away. I need my pain."
—Kirk, STAR TREK V
Similarly, I think of all the mistakes I've made in life—the big and the small. Yet I would not undo any of them because I have learned from all of them. They are part of how I got "here."
Oh, I agree, but if we assume that Spock is removing the actual memory of her, then the crime becomes so much worse.