I just watched III and IV for the first time in many years, a few days apart. To me IV is just not aging well, relative to III.
The rhythm of IV is misshapen. The threat-to-Earth scenes at the beginning feel like they go on forever, with needless insertions (sorry, Majel) of a character called simply "Chapel" saying lines anyone could have said. The middle part is lots of fun, but Kirk has just lost his son and shouldn't be having so much fun (I'm well aware that in the real world 2.5 years passed between theatrical releases of III and IV). And the whale/probe conversation also seems unending.
I am not about to start reediting the movie, nor would I do so without being able to replace the music in any event. But somewhere in there is a movie I would actually enjoy seeing again.
By contrast, I found that I could forgive III its weaknesses. First-time director, middle chapter of longer story, etc.