For me, it was not necessarily the level of destruction. This is the type of thing we see in the comics. It was meant to be horrifying and it was. What was missing though was some attempt on the part of Superman to prevent the destruction. So, when he saves the one family at the end, we are left thinking why is he saving them and not the thousands of people he just let die.
For me it was both. Mainly it was the callousness of having all this destruction without Superman doing a thing about it, but it was also the duration of it. When the city-smashing had finally died down and we had a lull and then Superman and Zod started going one-on-one and buildings started collapsing again, I was yelling "Seriously?" at the screen and very, very sorely tempted to storm out of the theater. It was just too much. Action films today are too driven to excess.
But also it was the sheer superficiality of it. You could remove all that devastation without it affecting the plot, characters, or dialogue in any way. It had no impact on the story whatsoever. If you read a dialogue-only transcript of the film, you probably wouldn't even know that most of Metropolis had been destroyed. It wasn't even part of the story. It was completely gratuitous, tacked-on spectacle.
Not to mention that it was stupid. Skyscrapers don't collapse like houses of cards the moment something slams into them. They're designed not to fall down. If Zod tossed Superman through a building with superstrength, sure, he'd punch a man-sized hole through the building, but the building would probably survive.
So on multiple levels, from plot to character to story logic to basic physics and engineering, that destruction simply should not have happened. And yet it dominated the film for a huge amount of time.