Kirk has lost his ship, his co-workers, his family, his world, and he's in danger of losing the one person he's brought with him. With all that on his mind, I don't care how nice Edith Keeler is, it just seems as if his attention and his emotions would both be fully occupied elsewhere.
I see it more akin to a soldier being in a war zone thousands of miles from home. He may not see any of the things he loves ever again so he takes comfort in the things that are around him.
I can see that being true if the person stuck in the past was Sulu or Chekov or someone like that. But Kirk isn't just some random solider; his being there is like having General Patton stuck in the past -- he's not going to adapt to his circumstances, he's going to change those circumstances so he can go home.