"Instead of a dead moon a living, breathing planet."
Because the Presentation Tape was a simulation, made years earlier, of how the effect might work when directed at a dead moon
. But, spurred on by the success with the Genesis Cave within Regula, Reliant's crew had been asked to find a "completely lifeless" planet.
The script states that the effect we see is "the 'Genesis Effect' WE SAW in its earliest experiments, now seems familiar on a gigantic, cosmic scale. The blinding flash at first, followed by the same tumbling turbulence of the miniature test. It is awesome."
The torpedo was not
meant to be detonated inside a nebula. The wave was not constrained and its effects were bigger and unpredictable.
Another thought: if Regula was simply transformed, why not keep calling it Regula, instead of "The Genesis Planet"? What are Klingons doing poking around Regula, presumably in Federation space; doesn't it make more sense that they are trying to check out why a huge nebula disappeared?