"There's no civilization here to interfere with" is pretty good reasoning. I suppose even Picard would have been moved. Doctor "Let's figure out if they're at an evolutionary dead end lest they interfere with the evolution of other species" Phlox...I'm not so sure.
A couple of other things I noticed after reviewing the episode.
1) Kirk goes even BEYOND explaining their own documents to them. He says the Constitution "Applies to The Kohms or it means nothing." Poppycock. The Constitution doesn't protect conquered people. But of course the Yangs are going to accept James T. Skygod's interp. BLATANT interference and....
2)...right in front of Captain Tracey! Who is one of the most conniving, devious, smart bad guys seen in Star Trek. Tracey would have Kirk over a barrel.
3) Wow, Kirk and Tracey are assuming a lot when they start their "Don't your holy texts say..." bit. I don't THINK it says 'Bible' on the front of that book. I don't know if that would be more unlikely than an identical Constitution appearing or less.