Why was the jihad inevitable? Because humanity had stagnated. After ten thousand years of galactic empire, people were not expanding. People rarely moved from one planet to the next. There was no advancements in art or literature or philosophy. Yet on this singularly important planet lived millions of people who had been oppressed for so long that they were ready to explode. The human race had been asleep for so long, but it was awakening, and about to test its limits. Paul awoke the sleeper when he showed the Fremen that they could challenge an empire. Nothing he did after that point could stop them from taking out millennia of wrath on humanity.
Why did Leto have to do what he did? Because despite the jihad, mankind was still stagnant. They were not expanding into the rest of the infinite universe. Someday, something (no, not thinking machines dang it!) would come and wipe them out. Leto oppressed mankind for five thousand years, pushing them into a crucible that was even worse than the previous ten thousand years of stagnation. When he died, humanity awoke again, realizing that there was a whole universe out there to be conquered. By the time they faced the challenge that Leto had foreseen, they had expanded too far and grown too strong to ever be fully defeated.
That is how I read the books, at least.