when Kirk wonders about the thing's ability to take a pounding, he already knows it is "very small" ... it does seem logical for him to think that a thing less than a hundredth the size of Kirk's ship should be easily destroyed ...
But how many photon torpedoes does it take to destroy a TOS starship? In the teaser to "Errand of Mercy", the Klingons pound the ship with half a dozen sucker punches of what very much look like photon torpedoes, and Spock reports "minor buckling" only. ... The problem is with the later, movie or TNG era portrayals of photon torpedoes as somewhat more effective in the ship-killing role.
We don't need to think of "photon torpedoes" as a constant unit of power, with the same destructive capacity no matter what ship fires them. In the era of the Horation Hornblower stories, ships carried cannon or carronades, but their effectiveness varied. Smaller ships carried smaller guns that shot smaller projectiles; larger ships carried larger guns that shut heavier charges. A big ship might carry 32-pounders, a small ship 12-pounders. You would talk about the "weight of a broadside", ie the weight of the shot fired from the guns times the number of guns.
So you can imagine a situation where a big ship is fighting a small ship. Both are firing "cannon", but the small ship's weaponry doesn't hurt the big ship, while the big ship can blow the smaller one out of the water.
I think it's useful to think of photon torpedoes in the same way. Maybe they are an energy weapon charged by the ship's power; so the their destructive capacity varies with the engine power of the ship firing them.