What most non gamers don't realize is that they are probably gamers too. Who do you think is buying shit like Angry Birds and Candy Crush? Most people who refer to themselves as gamers probably wouldn't touch that stuff with a 10 foot pole.
I don't really payu much attention to anyone who decides to look down on me for gaming. At the age of 41, I've decided that my generation was exposed to the first video games at a very early age and I've accepted that it is a way of modern life for me. For a good while there, I did think that I had outgrown it... but after watching my teenage son play Grand Theft Auto IV, I realized that this game was actually acted out by real professional actors.. which meant that there had to be a director... and that a lot of effort was put into this product to make it seem authentic. I've been an avid gamer ever since and I've actually grown to look down on mediocre forms of other media.