I don't know why ROTJ gets so much flack. Sure, the Ewoks aren't great, but besides that I consider it to easily be as good as Empire and A New Hope. When it comes to the OT, I pretty much like them all equally.
The big thing for me with ROTJ is that it's considerably uglier than its predecessor.
Take this shot from Empire:
Check the diffuse lighting that makes the fog seem to glow slightly. There's no clear light source. The ship is massive on the right, but its shape ends suddenly where the nose enters the water, forming a sharper angle. Luke steps uneasily towards the tip, and towards the background, walking to the left, which also makes him appear less confident.
There are three points of warm color in the image: R2's light, the light on the wing of the ship, and Luke's uniform. In any case, Luke is walking away from the lights. The small patch of ground exactly mirrors the angle of the ship's nose, but it's pushed off to the left and further away. It's unclear where the water ends and where the fog begins. The overall composition is an X or hourglass shape. Everything points to the water. It's just great composition.
But then, looking at Jedi, depicting roughly the same location:
Rather than having diffuse lighting, the scene is lit by multiple fairly direct light sources. The shadows cast in the fog by the ship's guns reveal at least two, pointing in different directions. It's basically just a plain white light. R2 glows bright white too, in what is supposedly a foggy nighttime scene. A subtler, yellowish light illuminates the background. The composition is quasi-rectangular: large quasi-rectangular patch on the right, quasi-rectangular pool of water, etc. None of the lines lead your eye anywhere. It's just not a good visual.
And this is one of the worst shots in the entire six-film saga.
You can chalk up part of the look to Marquand / Hume switching from the 100T stock used on Empire to Eastman 250T 5293, which was a short-lived, low-grain stock, but the (terrible) composition still speaks for itself.