The Enterprise was violating their territory in spite being being explicitly told to stay away. As soon as they ignored that then everything they did was illegal. No amount of hiding behind orders and regulations can change the fact that Starfleet and the Federation were totally wrong.
And what was Fox's justification? He wanted to establish a Treaty Port. Sounds innocent, doesn't it?
Treaty ports are essentially ports opened to foreign trade against the wishes of the host country, usually by threat of attack by a stronger power. "We need/want a port here and you're going to give it to us regardless of your own wishes.."
So much for the Federation and their much touted non-interferance policy.
I would imagine that shortly after the Enterprise left that the war started up again, this time with real, as opposed to virtual, weapons.
We may not understand how their virtual war worked but it was their system and the only casualties are those ships that ignored the code 7-10.