That is indeed the mindset. Perhaps even the nerd mindset, to borrow a phrase from another thread.If you approach an adaptation with the mindset that it * has * to be like its source material, you've automatically set yourself up for instant disappointment because the entire point of adapting something is to change the source material in some way.
I use to labor under such a mindset. I use to subscribe to. I use to loathe Starship Trooper's the movie. I hated the Baby Blues series and I couldn't stand Ben Hur. But, then I realized that I was just making myself miserable by being angry, rather than trying something new. Lord of the Rings introduced that to me, and I am grateful, extremely so, to the Jackson films for giving me the opportunity to learn what adaptation is all about.
Also, on a related note, there were African people in Britain during the Roman Empire. So, the possibility exists for Black people in Middle Earth moving around.