Some episodes have even said the Prime Directive is supposed to protect us as much as it is to protect the alien cultures. The problem here is that in the case you described, the reason for upholding the Prime Directive here is strictly out of our own benefit. We don't know what this culture might be like in the long run, so better to let them die rather than having them come back as our enemy.
The big fundamental problem is that this is merely an assumption, not a guaranteed fact based conclusion. Our characters will simply resort to the worst possible conclusion rather than take a chance that nothing bad will happen at all.
And if you want an even worse example of the Prime Directive being used BEFORE it was even created, check out the episode Dear Doctor.
The ship's doctor manages to convince Archer that it is better for a whole entire species to die out because it's a natural occurrence that they shouldn't interfere with. What makes it worse is that this species was seeking help by risking space travel without warp technology. And it's because of the simple fact that they don't have warp technology that everyone thinks they don't deserve help. Think about it. We have Vulcans helping humanity after we've created warp technology, even though we have just come out of a world war that killed 600 million of their own people, and the man who created it was a drunk who only DID IT FOR THE MONEY. But this race that developed space traveling technology so they can find help? Nope.