The biggest problem is why no one stopped off every so often to see how a potential threat was doing, by WOK Starfleet seemed to forget they were in that system (and hadnt noticed a missing planet), any Starfleet ship surveying that system should have had orders to scan the planet.
This is one of the points brought up in IDW's good comic, Khan: Reigning In Hell which bridges the gap between "Space Seed" and TWOK. Khan was a tyrant in many respects, yet Kirk and Scotty acknowledge that he had a sense of honor other respects. He committed no massacres during his rule and didn't attack anyone offensively, unless he was directly attacked.
Khan's enhanced memory made it easier for him and his followers to nearly succeed in controlling the Enterprise, and Kirk reasons in RIH that the same could potentially happen to another vessel or a starbase. That advantage plus the eugenics' somewhat ruthless nature ensured that they would make such attempts.
Khan, as per RIH, had a measure of respect for Kirk for genuinely defeating him (not easily done) and offering him a measure of honor in giving them another chance on Ceti Alpha V, even if it was barely charted and a challenging environment according to some other sources. He assumed that Kirk would indeed have some way of monitoring him if only for Starfleet's security, given the events on board the Enterprise, and that they might even be given aid solely because it was an ethical thing to do. That they would be ignored and left to survive on whatever was left did much to wear away any sort of respect or trust Khan might have had in the years building up to TWOK. Some of his people questioned his apparent faith in Kirk and led a revolt which ultimately cost their lives alone with Marla's, and Khan saw that as Kirk's blame too.