Yeah a non-political work still gets into murky waters when you have all the good guys as white and anyone with a darker skintone works for the Lord of Evil. I don't think LoTR is totally ruined by racism or something (especially as Sauron's human allies are mostly a sideshow) but it's got some issues we'd be rather critical of if someone tried to write that kind of thing today.
I don't think fantasy writers have blanket immunity from any responsibility to be careful around social issues. Especially in settings that draw on real-life places and cultures. Fantasy worlds like GoT still have something of our world, and thus can't help reflecting something of the writers views of the world. And images and ideas they put forward can shape in turn how we think about our world.
re summer islanders. I've only just started on book four and have hardly seen anything of them so can't really comment too much. One in a a feather cape startling some people who have probably barely ever even met a black guy doesn't sound like a big deal.
re: essos in more general, it honestly looks like rather a pick and mix. The Dothraki look pretty savage but we frequently see plenty of savagery in westeros, and through the Hound's opinions we get a critique of the hypocrisy of western knights. ie they're meant to be morally upstanding heroes but they're really just violent guys with swords. Dany meanwhile is playing white liberator of non-white cities but as I understand it that goes really really badly
so the narrative is hardly endorsing her.
tl:dr: I reject Dalekjim's "it's only escapism" as a cop-out but GoT hasn't really thrown any major red-flags at me so far.
It would be good to see a non-white culture that isn't based around either loot'n'pillage or slavery. But again I think one of the key points here is that Westeros looks more civillised at first glance but is in fact full of its own kind of barbarity. I mean, burning and murdering of the smallfolks seems a fairly regular pastime for warring lords. The riverlands are pretty fucked and all. Also the summer islands are supposed to be very peaceful, as I recall?