Maybe he watched B5 pre-Apocalypse.
That was the first thing that came to my mind, as well. No pun intended.
I'm trying to come up with a reason why he would have them. A reminder of what he's fighting against? Some of "his" people? Experiments from Milton? And what was with the head at the top? All the rest were pretty zombified, this one wasn't.
As for killing the army guys, I figure that it's not just he's concerned about them usurping his authority, any single person could try that. But this is an organized group with likely more loyalty to each other than they would have to him or his people. And they're armed, and young, and he's got quite a number of young-ish women who don't participate in defending the community but instead need defending. Possibly from young armed men. I can totally understand that, while he loses the muscle of the soldiers, he also eliminated the threat.