That's what's so confusing! He says he loves me, and when we're together he certainly acts like it...but then, all the bad shit gets in the way. What if I'm just not worth paying any attention to? That could definitely be the problem...
If you weren't worth paying attention to, then why does he pay attention to you at all in the first place?
If he pays attention to you when you're together, but seems to forget you exist when you're apart, then it's possible he's just using you to fulfill his own needs. Or he's just really shitty at having consideration for another person.
I honestly think we're just plain run out of things to say to each other, already....as sad as that is. In the beginning things were so great. I'd never been happier in my life, and let me tell you, I'm literally incapable of happy. I'd never had a guy make me feel the way that he did. But now......*shrugs*
It's better to find that kind of happiness in yourself and then find someone to share it with, rather than rely on someone else to make you happy. To put it another way:
Where you are: "This person loves me. There must be something that makes me worthwhile! But if he doesn't love me, then I'm not worthwhile."
Where you should be: "This person loves me because I'm worthwhile. If he ever fails to recognize that, I'll leave him for someone who won't."
Easier said than done, of course. Just don't rely on others for your happiness--it will always lead to disappointment.