It's not stretching at all. It's always been the very obvious endgame of a multiverse saga that culminates in an adaptation of Secret Wars. If they weren't looking to make a movie absolutely filled with cameos, references, variants and triumphant returns of popular actors/getting various popular characters who never got to meet each other in the same scene, then they wouldn't have gone down the multiverse saga/Secret Wars road in the first place. That stuff is literally what Secret Wars is.
There's never been any guarantee of any one actor actually saying yes, but you can be damn sure that every major actor who's ever starred in a Marvel movie were on MS's hypothetical wishlist for that final movie, if not for another one earlier in the saga. And while some of those actors might not get asked because enough people higher up the list said yes, Downey and Evans were always going to be the at the very top of that list.
Downey also did not need to cost them that much money. They could easily afford to pay his rates for a day or two of shooting for a minor, but impactful cameo and there's plenty of reason to believe that would've been the initial idea. This is a pivot, obvioiusly, because it's suddenly a massive role that is going to cost them a lot more than a hypothetical day or two of shooting and because casting him as Doom is definitely out of left field. But he was always going to be asked back one way or another. That much has been very clear since Secret Wars was announced.