I really don't care either way. Never really watched wrestling or golf myself. Golf is just too boring. As for wrestling, I never really had an interest in it anyway, but.... I think that last part nails it. Most of the other Olympic sports that I can think of are pretty simple to understand. Sure, there are occasional calls and such that the layperson won't understand, but they still "get" most of what's going on.
Well bear in mind that these sports were voted out. Now of course we don't know the exact numbers for the vote, but if some of these sports are hugely popular outside of N. America and Western Europe, it would seem likely that their would be enough votes from IOC members from countries outside of those to out vote them and keep them in.