Probably not, though. If you walk on somebody's lawn, saying "I never signed those STAY OFF THE GRASS signs and never paid for the barbed wire, either", and that somebody punches you, it's you who will get jailed for punching back and hospitalizing the somebody. It's no more self-defense than a bank robber firing at the armed guard who shouts "Freeze!".
If the nutjob next door tells you "I will kill anyone who treads on my grass" and your two teenage sons ride their bikes on his grass just to irritate him, has he the right to kill them? I mean after all he's warned them and you.
Look Balok and the Gorns just fired on the Enterprise and Federation outposts without discussing it with anyone. McCoy may have come to think the Federation was in the "wrong" but I still think the Gorns were. They didn't give the people on the outpost a chance. Sure it may be Gorn space, it may be free space, if in doubt don't kill people use Fox and his like to negotiate treaties. Fox may be a pompass ass but his intentions are honourable.
In the end the Federation may have accepted that they weren't reall wanted near Eminiar or Vendikar but for all we know the Vendikarians were keen to speak to the Federation