The problem is the phrase "I shouldn't have to concern myself." We all
should try to concern ourselves with other people's feelings. No, we shouldn't bend over backward to accommodate rabid spoilerphobes, but we shouldn't be completely inconsiderate about it either. Just because some reactions are unreasonable, that's no excuse to say "I'll act however I want and everyone else better stay out of my way."
Extremes in either direction are unhealthy. The middle ground is what we should aim for. I have little patience for the fanatical spoilerphobes, but I also try to respect more reasonable concerns about spoilers. Because that's just about basic consideration for other human beings. Sure, sometimes you're going to reveal something by accident because you can't be sure who knows what, but that doesn't mean there's no value in trying, in applying some general common-sense guidelines and consideration for other people.