Well, it would be easier for Finney to hide in the secondary hull, assuming that's where sections 18Y through 23D on B deck are located, in or near Engineering, were the ion pod located on the secondary hull.
Plus, we have seen parts of the saucer section really crowded, both during alerts and normal operations, but the vicinity of the shuttlebay seems deserted as a rule - a great way for Finney to make his getaway.
The farther the ion pod is from the bridge, the more plausible also the relative inattention Kirk gives to the pod during the crisis, and the more natural the fact that a senior officer would need to be detached there even when Kirk himself literally has the functions at his fingertips.
Also, something that looks like a lightbulb certainly meets our expectations of what a "pod" might be in the TOS visual context...
Back in the sixties, the ion pod might have been a fairly odd McGuffin, probably akin to a crow's nest in the mainmast; the need to jettison it never made much dramatic sense. Today, we can readily associate it with the practice of tornado hunting, and interpret it as a device to be delivered in the very middle of the storm; jettisoning would then be an obvious part of the mission, indeed its very purpose, and the button for this would justifiably be right beneath Kirk's index finger. Whatever was being done with
the pod prior to jettison would be secondary, mere preparations which Kirk would expect to be handled without delay by a man he trusted.
At some point between the first airing and the emergence of the treknobabble interpretation obsession I'm a prime victim of, the idea developed that the pod was a danger to the ship and for this reason had to be jettisoned. But nowhere in the episode do we find dialogue suggesting that the pod would pose a risk. All the angst is about the storm, and the changes in the storm that affect Kirk's choice of jettison moment. An "egg" to be laid by the ship at the best possible moment, a moment the CO has to choose by balancing scientific gain against the risk to the deploying ship, is an attractive interpretation there.