*pulls out Avengers Blu-Ray (as if I needed an excuse...)*
...and if a building collapsed in The Avengers, I don't recall it.
At 2:01:16, right inbetween Widow and Selvig discovering Loki's scepter and Stark asking J.A.R.V.I.S. if he is familiar with the tale of Jonah. You see a Leviathan plow through a building while chasing Thor and you see the building begin to collapse, but the scene cuts away before it collapses completely.