To be fair, there have been many cases of labels which initially were not intended as slurs initially which dropped out of polite lexicon. Many clinical terms for people with disabilities (dumb, lame, idiot, imbecile, moron, and more recently retarded) became associated with general insults, to the point that the original origins of the terms have been forgotten by most people. Then there's the category of antiquated racial terms like negro, mongoloid, oriental, etc which seem to have fallen out of use even though they weren't explicitly seen as racist by most. Or there's the complicated case of (American) Indian/Native American, where the latter seems to have been more or less imposed by well-meaning white people even though the community as a whole preferred the former appellation. Kind of similar to how Latinx is increasingly used even though 90%-95% of people of Hispanic ancestry don't use it personally (kind of odd we don't use Hispanic if we don't want a gendered term, because it's not actually a Spanish word and thus has no grammatical gender).
Over the course of my life, I've seen the term homosexual go from describing all people who have same-sex attraction to describing men in particular, to increasingly being frowned upon as being "too clinical." I wouldn't be surprised if it becomes a somewhat taboo term by the time I'm an old man.
