It's not the place of a Federation captain (or any military officer) to act as judge, jury, and executioner. You might end up there but it's really supposed to be a decision you reach without another reasonable choice. The fact that your enemy has committed greater war crimes does not justify you committing a war crime yourself.
Yes sometimes it is the job of a Starship Captain to act as judge, jury and executioner.
Plenty of times in TOS Kirk made arbitrary decision based on his experience and knowledge as a Federation Captain. As soon as he found a method he killed those flying things on Deneva. He killed that giant amoeba thingy. Also the cloud creature in Obsession, the Zetarians. Then there's General Order 24.
He even made the decision to strand Gary Mitchell. Even though at the time Gary had committed no crime.
And while nuKirk didn't have PrimeKirk's years of experience its not that hard to work out you don't let a mass murderer go no matter what.
You just hope that they always pick the right person for the Captain's job.