what instrument was keeping them afloat at the low altitude of 200,000 km up to that point?
Plot, I guess.
As for the TWOK issue and stranding him on Regula One, I went through this entire thread and I don't think I saw anyone actually quote Khan's speech to Kirk (paraphrasing from memory):
Khan said:"Perhaps I no longer need to try. [beams up Genesis]. I've done far worse than kill you, Kirk. I've hurt you. And I wish to go on hurting you. I shall leave you as you left me -- as you left her -- marooned for all eternity at the center of a dead planet, buried alive."
So, yeah, he changed his mind because he realized he could grab Genesis and no one would know he had it. Then he could learn how it worked, build an arsenal of Genesis torpedoes (maybe 72 of them...), and in the words of NuKhan: "... continue the work my people started centuries ago."