it would never be jettisoned until they knew for a fact the man was out of it.
But the fact that Finney stayed behind was really unexpected, and becomes all the more unexpected if we assume that Finney's role was merely to fire up the experiments in preparation for launch. After all, Finney had already given an "ion plate readings in progress" report, suggesting in this interpretation that the bulk of the work was done.
That Kirk would press the Red Alert button would be mere formality in a situation where Kirk was already certain Finney was long gone. "Being gone" is not a time-consuming process, as Kirk swears he gave Finney the needed time "and more" even though we indeed get the impression there was factually very little time between Kirk's real Red Alert keypress and his real Pod Jettison keypress - seconds or so.
That Kirk isn't in constant verbal communication with Finney, and has mission-specific buttons rigged to his control console, both indicate an atypical situation where Kirk for a rare once does not have time to speak to people. It's not that he doesn't care about Finney. He cares about the 430 lives aboard, but trusts that everybody will be safe even when he isn't constantly giving verbal orders and asking for verifications.
In general terms, we learn that Starfleet is quite willing and indeed rather desperate to send a starship straight into a storm that will seriously damage her and can destroy her. The mission to gather information from the heart of the storm appears to be a pressing one, and Kirk would be strongly motivated not to withdraw until he had deployed the pod. Getting out to check on Finney's welfare and then going in again seems out of the question, both because multiple penetrations would pose multiplied risk to the ship, and because the storm might go away. After all, if the ship has to plunge in before the pod is even properly primed, clearly the mission is extremely time-critical and access to the storm is only possible within a very narrow time window.
So, a mission to deploy a pod is something of a coherent whole (in other words, a positive sort of circular argument), even though other interpretations are possible and this particular one probably never occurred to the writers.