But by that logic you would have to headcanon in a refit between the Cage and Discovery too. Surely it is less of a headache to accept we are looking at the same Enterprise each time just through different lenses.Recasts don't fall into the calculation of what's canon on-screen since often they are unavoidable. However, SNW could have used TOS sets if they wanted to (as seen in TNG and Enterprise). Thus, a refit between SNW and TOS is canon.
Until Picard we just accepted that all Romulans had forehead ridges, we've accepted Trill and Bajoran redesigns. Surely a ship not looking like a 1960s set isn't that much of an ask. A show made today reusing those old sets just wouldn't have the mass appeal it would need to actually sell even the original ship design just wouldn't really cut it, it would look so dated and soulless. It worked well for one off nostalgia fun episodes but not for a series made in the 2020s.
I am curious how you reconcile Voyager's conflicted crew compliment. It was stated multiple times there were 143 crew and B'Elanna says there are 140 humans when we have 3 Vulcans, 2 Bolians a half-Klingon and half-Ktarian at least. These are on screen statements ergo canon but they cannot co-exist. If you are able to look past something like this then why not the visual representation of the Enterprise? If not then I feel sorry for your headaches.