Remember that Khan wanted to use Genesis as a weapon. The destructive potential of Genesis is that it must first destroy everything within a given radius before the terraforming process begins. That flame effect in the early CGI is the destruction process. It's just that the simulation was on a rocky planet, not an inhabited one. Other than the proto-matter flaw which was introduced in III, it's basically a nuclear weapon with positive after-effects, thus negating the MAD principle.