Hello! Stumbled upon this while writing a Foundry episode for Star Trek Online.
Some information I decided on for my story, purely non-canon but worth adding to the pile of speculation:
1) It's not true neutronium, otherwise the whole thing would have imploded. I came up with the idea that it's a quasi-degenerate mater similar to neutronium. Horrifically dense and impervious to most forms of energy.
2) The Doomsday Machine as we saw it was an eroded hulk, the external structures and controls long since eroded away. My pet theory runs along the lines of Peter David's speculations in Vendetta and other novels -- that the thing was controlled/managed by a race of beings with mind powers. Their last act for whatever reason was to set it to autopilot.
3) Torpedoes can't be tuned and tweaked as well a beam can. I've seen industrial laser cutters demonstrated in the past -- they alter the frequency and the amplitude of the beam in addition to the power settings. Add some technobabble to that and you can see why they were using the phasers. They were trying various adjustments, pulse-schemes, frequency rotations and so forth.
4) Antimatter "deactivation" -- powerful subspace fields disrupt the antimatter containment of approaching ships. Based on some speculation over the years, the Connies didn't have antimatter tankage beyond what they needed for a given encounter... "Day Tanks" or "Use Bins" rather. Antimatter was generated by some means onboard... subspace particle accelerator, quantum-charge reversal or some kind of zero-point trickery. This was abandoned as engine efficiencies made it practical to store fuel in large quantities. Anyway, "deactivated" refers to a safety shutdown of the antimatter plant and venting of the day-bins. This also neatly explains the no torpedo bit... No antimatter, no torpedoes.
5) Finally we get to the killing of the machine. I speculate it happened because of a few things... First of all there was a tremendous subspace/EMP effect as the ship detonated. In close proximity like that, it could have been enough to overwhelm the system and force a shutdown. Second I'm not discounting the fact that all those debris and chunks. SOMETHING got smashed or shotgunned by liquid metal. A far more energetic loss-of-containment event on the Yamato years later produced identifiable hunks, chunks, debris and a flaming saucer section.