It might be enough to get started, and hope you pick up a groundswell of popular support as you go.
Sure, their army isn't large, but since a VAST majority of the global population is dead, gotta think much smaller. It's not taking over the 250 Million plus of today, it's working on a small fraction of that, spread out over a big distance. No cars/tanks/planes, tv or phones, so you can work smaller areas, campaigns would take years to wrap up.
It took the Romans 40 years to conquer Britain. However, the Patriots have automatic weapons I'd think which is a game changer.