In general, yes, I agree with you. But within the MCU, I think there are reasons this doesn't work.
I know you're a writer, so I'm sure you're aware of the whole promise/payoff issue in fiction. To broadly summarize for others though, your readers/viewers will start the story with certain expectations based upon what is established at the beginning of a story.
See, the problem here is with the basic definitions. The MCU isn't one story. It's a shared universe encompassing multiple individual series, like Star Trek or Star Wars or the Arrowverse.
The problem is that people have started thinking of stuff like Infinity War/Endgame as the rule instead of the exception. Like I said, they don't get that Phases 4/5 aren't continuing that level of interconnectedness, they're going back to the looser approach of Phase 1 or the Netflix shows, where each character's stories pretty much stand on their own with only loose continuity between them.
So if you write a grounded crime fiction novel, for example, and then bring in demonic possession in the third act, you will alienate a lot of readers.
See, that's exactly why your analogy doesn't work. The MCU isn't one genre any more than it's just one story. You wouldn't expect to see Daredevil battling demonic possession, but you would expect to see Doctor Strange doing so. You wouldn't expect to see Star-Lord and Drax battling street-level criminals, but you would expect to see Luke Cage doing so.
The MCU has two issues here. One is fans have expectations, both based upon the Infinity Saga, and based upon the comic books, that will color their expectations. You may argue that that isn't fair, and I'd agree! But it's there.
The job of fiction is not to limit itself to people's expectations, but to challenge and shatter them.
The bigger issue is that the MCU continues to use foreshadowing, repeatedly. This is most commonly done through the use of mid and post-credits scenes. In Phases 1-3, these almost always (unless they were harmless, comedic fluff) foreshadowed future movies. I think the only one of these guns which didn't fire was the scene at the end of Spider-Man: Homecoming involving Mac Gargan (who was intended to be Scorpion). In contrast, almost every post-credits scene in Phase 4/5 has yet to mean much of anything. The only one from the movies which actually foreshadowed anything was the Black Widow scene (which led into Hawkeye). Also, I guess, the Loki Season 2 trailer tacked onto the end of Quantumania. There were a handful from the Disney+ shows too - WandaVision teased elements of both The Marvels and Multiverse of Madness, notably. But most of the non-comedic trailer scenes have seen no payoff - even if they came out 2+ years ago - with payoffs not likely on the immediate horizon. Thus these post-credit scenes feel like pointless teases now that Marvel has no intention of going through with in the next five years, if ever.
I'll agree that's an issue, but it's a minor, incidental one. You're just underlining the problem, which is that fans have blown the supplemental continuity ties way out of proportion. People are more preoccupied with the brief post-credit tags and gags than with the movies themselves.
I'm not saying there isn't a level of interconnectedness. I'm saying that it's secondary to the merits of the individual stories, like it was in Phase 1. I mean, so what if there hasn't been payoff in two years? It took four years to get from Nick Fury's first mention of the Avengers Initiative in Iron Man to the formation of the actual Avengers.
Wakanda Forever, for example, was good, but it was transparent that the whole Riri element was played up in an effort to promote Ironheart,
No, it wasn't. I hate it when people say that, because it's misunderstanding how story and character work on a profound level. Riri was absolutely essential to the story of Wakanda Forever. She was a reminder to Shuri of what she used to be, which not only motivated Shuri to protect her but gave her the freedom to leave that role to Riri and move on to embrace the role of Black Panther. Riri was the only reason Wakanda and Talokan were even in conflict, because they probably would've been on the same side on the vibranium-detector issue if it hadn't been for Namor's determination to kill an innocent teenage girl who was an embodiment of the very African-diaspora youth that T'Challa opened up Wakanda to help in the first place, and thus an embodiment of T'Challa's legacy. Riri was exactly the character who had to be in that role in that specific story, regardless of what it was setting up for her future.
Again, you're making the mistake of seeing continuity and assuming it's the exclusive reason for a thing rather than a secondary aspect. Something can mean more than one thing at the same time, the fundamental and the harmonics.
and whatever the fuck they were doing with Allegra de Fontaine was to help set up Thunderbolts.
Okay, here I agree. I don't see the point of de Fontaine at all.
Hawkeye larded up what could have been a tightly-focused story on Clint/Kate with the needless Yelena stuff.
Hard disagree. Kate/Yelena was awesome. I mean, if Kate's inheriting the role of Hawkeye, doesn't it follow that she gets her own Black Widow in the bargain?
Quantumania was so focused on the introduction of Kang it forgot to give Scott Lang a character arc (and didn't give Hope anything to do at all).
Scott's arc was about his relationship with Cassie, about working through her resentment at his absence and coming to trust her as a hero. QM was really more Cassie & Janet's movie than Scott's, and that's perfectly all right, given that it's pretty much an ensemble series and those two characters didn't get as much focus in the first two.
So people have this expectation that it's all going to fit together because...Marvel built up the expectation. And continues to build it up, across most contemporary projects.
Again, though, just because there's continuity doesn't mean that's the exclusive point of it all. The parts don't exist only for the purpose of combining into a whole. Something can mean more than one thing.