Just how many actual fighters, and how many possible fighters, are at Woodbury?
They started with 72, added Tyreese and his sister and the 2 jerks (76). They lost 6 to Rick's group and 8 to Merle (62). They were men and women who seriously were more hindrance than help in a fight, and a number of children. Even if you limit that to 20--and I think it's more--that leaves 42 possible fighters. Half of which likely suck at shooting and fighting compared to Rick's groups, but even a fool can get lucky.
Ricky's group has Rick, Daryl, Glenn, Maggie, Carl, Michonne, and limited Herschel. Carol, Beth, and obviously Judith are not fighters. Carol can provide some support on moving weapons, reloading, etc. Beth needs to protect the baby. So maybe 7.
Odds at guerrilla warfare? Odds at ambushing in the prison? Cut off a large group and Rick et al have a good chance.