If we accept that Kirk fired on Nero because he honestly felt that he could escape through the wormhole then we also have to consider the probability that the same thing actually happened to Vulcan and Nero didn't actually kill everyone on the planet. Perhaps Vulcan was just shifted to an alternate universe and/or back in time. Reasonable doubt works both ways.
When Spock and Nero travelled back in time, they both entered a black hole that was created by red matter. When the Narenda was being destroyed at the end of the movie, the black hole was inside the ship. We saw it being bent, torn and crumpled. If the Narenda could reasonably be expected to survive passage through a black hole to the extent that Nero and his crew would survive then the same should apply to Vulcan. We also saw it being distorted. Perhaps a red matter black hole tears apart whatever falls into it but then reassembles it on the other side. Much like a transporter would reassemble you and not move you whole from one spot to the other.
If Kirk had reasonable doubt that Nero would die then Nero could use the same defense. Kirk had no proof that Nero killed everyone on the planet and then destroyed the planet. Never having encountered red matter black holes before the on;y thing that he could say for sure was that they transport objects back in time. For all we know everyone on Vulcan is fine, even the guy that got flattened by the statue. When you emerge out the other side, time runs backwards and you end up exactly in the same physical state you were when you first entered the black hole.
Except that we saw Vulcan crumble to pieces. Saying it was reassembled on the other side has no merit, since there is no way to test the theory. Yet even if that were a viable theory, for all we know Vulcan could have been reassembled in an alternate liquid universe, and everyone instantly died from severe radiation poisoning upon arrival.
All we know is that from what we saw, billions of innocents died, and the madman who did it so casually, tried to do it again to another planet full of billions, and had plans to further destabilize the Federation with untold billions of lives in the balance.
Even though the Narada was breaking apart, there's no reason to assume it wasn't making it out the other side, because it had traversed the singularity before, and had come out whole. A hypothesis could be made that small enough objects could pass through the singularity without being destroyed. There simply is no way to know, especially not when one is in the heat of battle, the enemy has refused to surrender, and has no intention of ever stopping his killing spree.
This means you have to go on the available data, which is:
* Nero is an absolute and imminent threat to all life forms.
* Nero's ship is extremely powerful, capable of destroying multiple starships without sustaining damage.
* Nero's ship passed through a singularity once, and may be able to do so again.
* Nero's ship is still an unknown quantity, and may contain weapons or devices as yet unknown.
* There is no time left to assess the possible capabilities of his ship due to being in a situation requiring immediate action lest your ship may be destroyed.
* Nero has rejected any form of offered assistance.
It's nice to play the what-if game, but at the end of the day, Kirk still took the only real course of action available to him.
