It would be as if we invaded a country in South America during WWII and claimed it was an important part of the war. Then the Nazis sent people to attack us there and we lost more troops.
EXACTLY

Might as well have invaded China in response to Pearl Harbour. No, scratch that. That would have made WAY more sense than the Iraq War, as there were actually Japanese troops in China.
Terrorists who were there because we were there. As pointed out already, Saddam's regime was one of the most anti-Islamic Extremism in the world. Fundamentalist Islamic terrorists would have found it extremely hard to operate within his borders. By removing him from power and opening up the country to an influx of Islamic extremists with a ready made, home delivered, US target, we created a war on terror there, we didn't solve one.