What was it Carol Marcus said... "There can't be so much as a microbe or the show's off."
Only in the sense of ruining the scientific validity of the experiment, though. Genesis would work just fine, but would not prove its ability to create life from absolutely no life.
After studying the Genesis material, Spock was confident that Genesis would "destroy [preexisting] life in favor of its new matrix". This would probably have been the fate of the Enterprise
crew as well.
it's basically a nuclear weapon with positive after-effects, thus negating the MAD principle
The positive effects would be to the attacker, though, not the victim - and MAD would still hold, as both of the attacking sides in an exchange of Genesis torpedoes would also be at the receiving ends, and would be destroyed in favor of the new matrix.