This may seem like out there theory to some of you, but I honestly think one of the reasons for Trek 09's success was that it had far better trailers
compared to ST: ITD.
The trailers for Trek '09 really pumped people up, I even remember plenty of non-fans talking about how they wanted to see the movie just by the trailers alone. I never heard the same sentiments from people who saw the STITD trailers. ITD trailers were nice, but they were not as epic and grand as Trek '09. That's partly why a huge mainstream audience flocked to see that movie.
Go to Youtube and compare to the trailers for Trek '09 to ITD,...Trek '09 is still stirring.
Here is a sample of Trek '09 trailers:
So I think it came down to marketing as well, they should have bragged about having Khan more instead of the big secret.
While I don't agree with the idea about bring out the fact that Cumberbatch was playing Khan (mostly because people would've been WAY more negative up to and after release), I do agree that most of the STID trailers sucked. Only the final domestic trailer one felt like it was actually interesting, since most of them kept rehashing the same footage over and over.