I think at the end of the day, the most basic reason for the contempt for Nemesis is that it let us all down.
So many bad choices where made, large and small, by so many people (though I think Stuart Baird as director has to own the lion's share of the blame, with Rick Berman as Trek's overlord as runner-up in the 'buck stops here' department) and... it was like a poorly woven tapestry in the end.
Brent Spiner wanting to be killed off as Data. Patrick Stewart's need for speed and action-hero status rather than recognizing and accepting his age the way Shatner did in TWOK (even if he later pushed back against such a bit). Riker and Troi's departure being present but not real throughout the whole thing. Worf's lack of an explanation for why he had suddenly come back to Starfleet. The useless inclusion of another identical-to-Data Soong type android. The way the Romulans rose as a specter of "yes, finally the Romulans as the baddie in a film!" only to be dashed the more we learned about it. The reveal of the Remans as... well, disappointing as a species as anyone can recall. Tom Hardy's performance (he's got acting chops, but at that time, he was a bit too new and likely had no direction to work from). The lack of any real character growth (that made it to film, IE deleted scenes).
I also think that one or two big key choices could have and should have been made up-front that would have saved the film: with the focus on Romulus, we should have seen a cameo at least of Nimoy as Spock, and we also should have seen Denise Crosby as Sela, whether in the Donatra role or something else.
The whole escapade felt half-hazard and slapped together, when by all rights it should have and could have been so much better.
Even the actors agree with this sentiment. They're all very quick to blame Stuart Baird, and I can't blame them for such.