When the Observers first developed time-travel, they went to 2167 and used their tech to erect a temporal "dam" of sorts. This is a safeguard to protect their species Origin Point (Feb 20th, 2167) from temporal ripples that might come should temporal incursion happen pre-2167 (where the dam would hold back the "waters" of temporal change). With 2167-2600s being essential to causally justify Observer existence, it is forbidden to travel within that period, lest someone disturb the flow and alter Observer ontology.
This would explain why the actions of the Science Team and the multiple timeline alterations have not affected Future HQ; the ripples were held back by the space-time blockade, and the changes were generally confined to the 1985-2020s. The Science Team was concerned with restoring stability, however, potentially because the Loop might risk cracking the dam.
Whatever the case, this also applies for the Invasion. Once the foundation has been set and the preparation of their new planet is finally complete, the remaining Observers in 2609 will relocate to the 21st Century, the 2167 floodgates will be opened, and the temporal ripples of a history where Observer succeed in their Invasion would finally spread beyond 2617, likely undoing Future HQ (but it had been abandoned anyway, so it wouldn't matter anymore). Another implication is that for The Plan to work, Michael/Walter will either have to destroy this space-time barricade, or jump on the other side, which would allow them to alter Observer history.
Clearly, if The Plan involves changing the outcome of 2167, then that date must "anchor" the causality of the Observers, so the counterargument that the Observers "are impartial to all timeline changes by virtue of their nature" wouldn't apply here. I've championed that idea for a long time, but it is evidently obsolete, now, and in this barricade scenario, it is more that the Observers have shielded their Origin Point, and so long as that is preserved, they can withstand any timeline changes. Something to think about.