After all a crewman's life is on the line and the decision to push the button would have to be made quickly.
To the contrary, I'd think that since a life is on the line, everybody else should keep their dirty fingers off the button, and Finney and Finney alone should have the authority to jettison the pod. Kirk would have the complementary authority to abort the mission, simply by flying out of the storm. Clearly, flying and ion pods are perfectly compatible things: Kirk e.g. orders a speed change while Finney is fiddling with the pod, and tells helm to hold course as if there were alternatives.
All evidence for the pod itself being dangerous or harmful to the ship, or even an inconvenience, is circumstantial at best. Evidence of pod jettison being harmful to Finney is more solid, as nobody really expects him to have survived the event - but Kirk does order a search in or around
the ship, which would be consistent if Finney were stuck in a chute accessing a pod that is about to be launched, the same way Scotty accesses instrumentation in that angled tube in many episodes, and might have been sucked out with the pod and blown into space. If Finney were in the pod in the sense of sitting inside at the observer's seat, one would expect that whatever emergency pressure door system saved the mothership from losing its air at jettison would also save Finney's life, and he would be searched for, not in or around the ship, but at the current location of the ion pod.
Fundamentally, of course, we have the problem here that the writer was most probably thinking about a crow's nest in the mast of a ship, or an observation balloon/kite, and then adding this jettison thing without thinking through the implications - whereas modern audiences can more comfortably think in terms of tornado hunters, and appreciate the idea of deploying and dropping monitoring devices in the middle of a non-survivable phenomenon in a hair-raising maneuver. And vice versa, modern audiences would be appalled to think in terms of sending a person to die in an observation balloon.
The writing, regardless of intent, caters for both interpretations. But it's not particularly formulated to support the crow's nest approach. Finney is never indicated to be in any danger before
Kirk supposedly murders him - the ion pod is not a dangerous assignment in any other sense than as a possible tool for a sinister plot. The ship in turn is indicated to be in constant danger, which Kirk constantly judges through his own expertise. We lack information on what happens when Kirk declares the danger unbearable: the computer tells the lie that he ejects the pod, while dialogue vaguely suggests Kirk actually first calls Red Alert and then fires the pod, then somehow saves the ship.