I think slapping the death penalty on visiting Talos IV is immensely logical.
Starfleet knows that the planet is a threat to the entire universe, as its inhabitants can take control of any individual who pays a visit. On the other hand, Starfleet knows that the planet would be an alluring target for people who want their share of lost Talosian treasures from the planet's past, but don't know the risks (or, worse still, believe the implausible Talosian lie that they are "no longer interested"). Yet Starfleet can't reveal the risks, because those, too, would be alluring - no second-rate Trek villain would skip a chance to steal a power that can subjugate the universe.
So basically Starfleet
has to kill everybody who has visited Talos IV anyway, but can't tell anybody why. A death penalty without an explanation is both a suitably exceptional deterrent (it always helps in getting your way if people believe you are an unstoppable homicidal maniac) and a help in streamlining the necessary process of executing anybody who would have been co-opted by the Talosians.
The only thing that could reverse this sensible policy would be... Well, the events of "The Menagerie". Kirk proved two things: that Talosians cannot be contained merely by executing people who visited the world (the fake Mendez, lightyears away from Talos, is proof enough of that!), and that despite now demonstrably being perfectly capable of taking over the universe, the Talosians cannot be observed to have done so (meaning either they really are harmless, or then they have
already won and taken over everybody and everything, and in neither case is any action warranted or possible).
I would assume that threats of such immense magnitude pop up basically once in a decade, at the rate Starfleet is poking at alien things. And that the threats subsequently are dealt with, so the mysterious death penalty deterrent can be transferred to the next disaster zone (just as we see it hop from GO 7 to GO 4 during the course of TOS) - or then everything is lost forever, Armageddon comes, and a bit of time travel is required to put things right again.
I watched Doomsday machine the other day and don't recall any mention of a death penalty
I believe the reference was supposed to be to "Ultimate Computer", where Kirk argues M-5 to death.
M-5: "No life."
Kirk: "Because you murdered it. What is the penalty for murder?"
M-5: "Death."
Kirk: "And how will you pay for your acts of murder?"
M-5: "This unit must die."
It should be remembered, though, that M5 is stark raving insane. Kirk is using this tactic on it because he has previously learned that M-5 is built on the memory engrams of Richard Daystrom, and he has just heard Daystrom argue that murder is wrong and naughty and against "civil and moral laws we've lived by for thousands of years".
It would be easy to argue, then, that Kirk is hanging M-5 on a noose that has otherwise been consigned on the dustbins of history - he just bets that belief in outmoded systems of justice is the crippling weakness of M-5, owing to it being a characteristic of Daystrom's thinking.
Remember that elsewhere in TOS, we saw a system for dealing with criminals that did not include the concept of penalty in any form. If you did wrong, you got brainwashed; if you did really horrible things, you got brainwashed with somewhat stronger detergents, and ended up like Lethe in "Dagger of the Mind".
Timo Saloniemi