It's heartening to see everyone in this thread (minus the OP) has come out in support of A&E and against Robertson's remarks.
The "black people were better off under Jim Crow" meme is a bizarrely common one in the South, it seems. Robertson also shows every indication of being fully absorbed in his privilege. "I didn't witness this, this didn't happen to me, so it must not have happened. And I don't get guys wanting to have sex with each other, so there must be something wrong with them."
If the show gets canceled as a result of all this, I won't shed any tears. I've seen a few episodes. It's fun, as semi-scripted reality shows go. The hook seems to be that these guys are smarter than they look. If so, it's apparently not by much.