I always wondered, why didn't Kirk just...take away the weapons the Klingons gave the natives? There were no factories or production houses anywhere, so they weren't making the weapons themselves.
1. The Klingons were teaching the natives how to make them.
2. You going to keep a ship in constant orbit to monitor them? Because nothing stops them from bringing more once you leave.