If Steve decides that it’s best for the future to let Peggy be manipulated by Hydra, he should just stay away. Marrying her, playing the part of the devoted loving husband, encouraging her to keep up the good work while he’s secretly sending her out into the world every day to serve Hydra, is disgusting.
Is it? Would Ms. "He'd tell you to do as Peggy says!" really appreciate it if Steve started playing God fresh off of his one-in-fourteen-million win, saying he knows better because he's seen the golden path that ends with 2023 having pretty much the same number of people as 2018 did, and he's going to try doing as little of that as possible. Keep things loose. Don't worry your pretty little head, honey, I know what decisions you should be making.
I mean, sure, I might dick around with an alternate timeline just to see what happens, but I'm a big subscriber to the Teal'c philosophy that our reality is the only one of consequence, and as you say, I'm not the one whose looking my wife in the eye saying, "Let's do this, it might backfire horribly and destroy the world, but don't worry, I've got a spare I can go home to where I know the all the things you would've done turned out fine. Not great, but, in the end, fine. Now let's kick over this anthill full of hidden Nazis and see what happens."
Like, in the real world, more than once, we were seconds away from total nuclear armageddon caused by embarrassingly stupid mistakes. If you send me back in time before détente, I'm going to be afraid to breathe. I don't want to sneeze on a butterfly that flaps its wings and then suddenly Stanislav Petrov has to call in sick on September 26, 1983. I don't know whether Steve Rogers has the heart of a gambler or not, but I'd think that after so many last-second victories, he'd recognize the world sometimes hangs by a thread, and you don't want to go tugging on it. At most, I'd think he'd throw some sand in the gears, set things up for the future victories so they had a little more margin for error, rather than just sending up a big flare saying "CRYPTO-NAZIS, MURDER MY WIFE AND ME RIGHT NOW TO PRESERVE YOUR SECRET."
Also, how do we know Peggy doesn't know about the Hydra infiltration? She was dealing with it back in the '40s, though she hadn't found the shape of it yet when we left her. We saw in this very film that she worked in the same building as Arnim Zola and his soon-to-be-computerized brain, and that her good, close friend Howard Stark was his collaborator. Maybe she was working against them the whole time. Maybe she found them useful and kept an eye on them to make sure they didn't get too big for their britches while still doing things loyal SHIELD agents wouldn't. Maybe she was turned, and would've murdered Steve in his sleep if she found out he knew.