We're nearly 10 movies into this franchise, we have almost 2 full seasons of a TV series, a mini-series and a full-season of Netflix Original in this universe -with more NO on the way. The universe starts to feel crowded at this point and almost conflicting in some regards.
Age of Ultron is MCU film #11.
1) Iron Man
2) The Incredible Hulk
3) Iron Man 2
4) Thor
5) Captain America: The First Avenger
6) The Avengers
7) Iron Man 3
8) Thor: The Dark World
9) Captain America: The Winter Soldier
10) Guardians of the Galaxy
11) The Avengers: Age of Ultron
This doesn't include the One-Shots
The Consultant
A Funny Thing Happened on the Way to Thor's Hammer
Item 47
Agent Carter
All Hail The King
...or the TV series
Agents of S.H.I.E.L.D.
Agent Carter
Daredevil
...and the forthcoming shows:
Agents of S.H.I.E.L.D. spinfoff featuring Mockingbird
Netflix's AKA Jessica Jones
Netflix's Luke Cage
Netflix's Iron Fist
Netflix's The Defenders
Basically, your point is well taken. There's tons that has happened by now that should be having huge impact on the world but is just being swept aside for some reason, likely to "keep the status quo" so the movies can keep being made.