I love The Matrix but I thought the sequels were empty CGI fluff and I haven't bothered with them in many years. I thought The Animatrix dealt with the thematic issues better, as well as the short run of comics that were produced (which, incidentally, introduced me to Neil Gaiman).
On the flip side, The Pirates of the Caribbean trilogy is often derided by movie goers, but I love all three (On Stranger Tides, however, is weak) and I still love At World's End the most. Things may have gotten unwieldy towards the end, but I've always found everything held up well enough.
I definitely agree about the Pirates. It's an interesting comparison, I think, to the Matrix, since both tend to suffer the same problem for what, I think, are at least some of the same reasons: great first movie raising expectations (perhaps beyond a reasonable height), sequels shifting focus in a way that not necessarily everyone liked, sequels being made as basically a two-part single story, which I know turned off a lot of people. But I think where the Matrix sequels somewhat undermined the world of the original and seemed to lose part of what made the original special in their quest for bigger and better, the Pirates sequels were much more consistent (albeit also much less ambitious). Honestly, I think if they hadn't spread the story out across two movies (and hadn't filled in so much of the extra space with a little bit too much of just Johnny Depp being weird) that the Pirates sequels wouldn't have an image problem at all. Except for Stranger Tides, which was just uninspired.