To understand the Organians and the treaty situation, you may want to consider these passages from "Errand of Mercy":
(with thanks again to Chakoteya.net transcriptions)
...and also this passage from Act 1:
...and then there's this from Act 4:
Given that Trefayne seems to know what the Klingons are up to, but only announces what he knows once the enemy fleet assumes orbit, we can assume the Organians are (mostly) concerned with how the conflict affects them on their planet. Ayelborne and Claymare seem to infer they have the power to paralyze the entire Federation and Klingon Empire, all at once, and are already doing so, but that they don't want to interfere. Most telling is that Ayelborne and Claymare seem to be complaining that the Organians find the "mere presence" of Kirk, Kor and company to be "intensely painful" asking them to leave. Spock's comment in the Teaser makes it clear that the hostilities have focused on "the only Class M planet in the Disputed Area, strategically located for either side", and the Organians' hot tomale/paralysis trick makes it clear neither side is going to have access to the planet, take the planet off the table and that diffuses the conflict immediately.
With the exception of subsequent mentions of the Organian Peace Treaty, we never hear mention of or see the Organians ever again. This tells me that the peace treaty is simply called Organian by the Federation and the Klingons because Ayelborne dictated terms before vanishing, Kirk and Kor reported what Ayelborne said to their respective superiors, and the language made it into a treaty negotiated by the warring parties without Organian participation. And I think the episode is clear that we are never expected to see the Organians ever again (at least not Ayelborne and company) as the characters deliberately transform into non-corporeal beings before everyone's eyes and are not heard from for the rest of the story. There is no disembodied "use the force, Luke" to imply these aliens are somehow still active participants after they disappeared.
So from all of this we can conclude that it is indeed possible that Federation and Klingon starships can do battle, or at least take an occasional potshot at each other. But both sides are probably worried that engaging in an all-out war, particularly with Organia apparently interposed between the Federation and the Empire, would risk of another confrontation with the Organians. This is obviously something both sides are deathly afraid of. With Romulans, Cardassians and other powers lurking about, neither side would want to find their fleets disabled for any period of time.
I also get the impression from Spock's and Ayelborne's comment in Act 4 that the Organians look upon humans, Klingons and the rest as if we were like bacteria or a fungus. They're capable of interacting with us, but they would prefer to avoid getting athlete's foot or diarrhea or whatever. So shoo... go 'way...
And both warring parties obviously knew this right from before the treaty was even written, as Kirk and the Klingons never hesitated to take up arms after that. Clearly, nobody was ever worried about Organians looking over their shoulders.
(with thanks again to Chakoteya.net transcriptions)
TEASER
[Bridge]
SPOCK: Captain, we've reached the designated position for scanning the coded directive tape.
KIRK: Good. (puts it into a decoder) We both guessed right. Negotiations with the Klingon Empire are on the verge of breaking down. Starfleet Command anticipates a surprise attack. We are to proceed to Organia and take whatever steps are necessary to prevent the Klingons from using it as a base.
SPOCK: Strategically sound. Organia is the only Class M planet in the disputed area, ideally located for use by either side.
KIRK: Organia's description, Mister Spock.
SPOCK: Inhabited by humanoids. A very peaceful, friendly people living on a primitive level. Little of intrinsic value. Approximately Class D minus on Richter's scale of cultures.
KIRK: Another Armenia, Belgium.
SPOCK: Sir?
KIRK: The weak innocents who always seem to be located on the natural invasion routes.
...and also this passage from Act 1:
(Council of Elders' Chambers)
KIRK: Mister Sulu, follow your orders. Get out of here. Contact the fleet. Return if the odds are more equal. Kirk out. Gentlemen, you kept insisting that there was no danger--.
AYELBORNE: That is correct, Captain. There is no danger.
TREFAYNE: Ayelborne, eight space vehicles have assumed orbit around our planet. They are activating their material transmission units.
AYELBORNE: Thank you, Trefayne.
KIRK: Can you verify that?
SPOCK: Negative, Captain, but it seems a logical development.
AYELBORNE: Captain, since it is too late for you to escape, perhaps we should do something about protecting you.
KIRK: If you had listened to me
CLAYMARE: We must be sure you are not harmed.
TREFAYNE: Ayelborne, several hundred men have appeared near the citadel. They bring many weapons.
KIRK: How does he know that?
AYELBORNE: Oh, our friend Trefayne is really quite intuitive. You can rest assured that what he says is absolutely correct.
...and then there's this from Act 4:
(Kor's Organian heaquarters)
KOR: My fleet, it's helpless.
KIRK: What have you done?
AYELBORNE: As I stand here, I also stand upon the home planet of the Klingon Empire, and the home planet of your Federation, Captain. I'm putting a stop to this insane war.
KOR: You're what?
KIRK: You're talking nonsense.
AYELBORNE: It is being done.
KIRK: You can't just stop the fleet. What gives you the right?
KOR: You can't interfere. What happens in space is not your business.
AYELBORNE: Unless both sides agree to an immediate cessation of hostilities, all your armed forces, wherever they may be, will be immediately immobilized.
KIRK: We have legitimate grievances against the Klingons. They've invaded our territory, killed our citizens. They're openly aggressive. They've boasted that they'll take over half the galaxy.
KOR: Why not? We're the stronger! You've tried to hem us in, cut off vital supplies, strangle our trade! You've been asking for war!
KIRK: You're the ones who issued the ultimatum to withdraw from the disputed area!
KOR: They are not disputed! They're clearly ours. And now you step in with some kind of trick.
AYELBORNE: It is no trick, Commander. We have simply put an end to your war. All your military forces, wherever they are, are now completely paralysed.
CLAYMARE: We find interference in other people's affairs most disgusting, but you gentlemen have given us no choice.
KIRK: You should be the first to be on our side. Two hundred hostages killed.
AYELBORNE: No one has been killed, Captain.
CLAYMARE: No one has died here in uncounted thousands of years.
KOR: You are liars. You are meddling in things that are none of your business.
KIRK: Even if you have some power that we don't understand, you have no right to dictate to our Federation--.
KOR: --Or our Empire!
KIRK: How to handle their interstellar relations! We have the right--.
AYELBORNE: --To wage war, Captain? To kill millions of innocent people? To destroy life on a planetary scale? Is that what you're defending?
KIRK: Well, no one wants war. But there are proper channels. People have a right to handle their own affairs. Eventually, we...
AYELBORNE: Oh, eventually you will have peace, but only after millions of people have died. It is true that in the future, you and the Klingons will become fast friends. You will work together.
KOR: Never!
CLAYMARE: Your emotions are most discordant. We do not wish to seem inhospitable, but gentlemen, you must leave.
AYELBORNE: Yes, please leave us. The mere presence of beings like yourselves is intensely painful to us.
KIRK: What do you mean, beings like yourselves?
AYELBORNE: Millions of years ago, Captain, we were humanoid like yourselves, but we have developed beyond the need of physical bodies. That of us which you see is mere appearance for your sake.
KOR: Captain, it's a trick. We can handle them. I have an army.
(Kirk holds him back as Ayelborne and Claymare transform into pulsating lights, too bright to look at. Then the Organians disappear.)
SPOCK: Fascinating. Pure energy. Pure thought. Totally incorporeal. Not life as we know it at all.
KIRK: But what about this planet? The fields, the buildings, this citadel?
SPOCK: Conventionalizations, I should say. Useless to the Organians. Created so that visitors such as ourselves, could have conventional points of reference.
KOR: But is all of this possible?
SPOCK: We have seen it with our own eyes. I should say the Organians are as far above us on the evolutionary scale as we are above the amoeba.
Given that Trefayne seems to know what the Klingons are up to, but only announces what he knows once the enemy fleet assumes orbit, we can assume the Organians are (mostly) concerned with how the conflict affects them on their planet. Ayelborne and Claymare seem to infer they have the power to paralyze the entire Federation and Klingon Empire, all at once, and are already doing so, but that they don't want to interfere. Most telling is that Ayelborne and Claymare seem to be complaining that the Organians find the "mere presence" of Kirk, Kor and company to be "intensely painful" asking them to leave. Spock's comment in the Teaser makes it clear that the hostilities have focused on "the only Class M planet in the Disputed Area, strategically located for either side", and the Organians' hot tomale/paralysis trick makes it clear neither side is going to have access to the planet, take the planet off the table and that diffuses the conflict immediately.

With the exception of subsequent mentions of the Organian Peace Treaty, we never hear mention of or see the Organians ever again. This tells me that the peace treaty is simply called Organian by the Federation and the Klingons because Ayelborne dictated terms before vanishing, Kirk and Kor reported what Ayelborne said to their respective superiors, and the language made it into a treaty negotiated by the warring parties without Organian participation. And I think the episode is clear that we are never expected to see the Organians ever again (at least not Ayelborne and company) as the characters deliberately transform into non-corporeal beings before everyone's eyes and are not heard from for the rest of the story. There is no disembodied "use the force, Luke" to imply these aliens are somehow still active participants after they disappeared.

So from all of this we can conclude that it is indeed possible that Federation and Klingon starships can do battle, or at least take an occasional potshot at each other. But both sides are probably worried that engaging in an all-out war, particularly with Organia apparently interposed between the Federation and the Empire, would risk of another confrontation with the Organians. This is obviously something both sides are deathly afraid of. With Romulans, Cardassians and other powers lurking about, neither side would want to find their fleets disabled for any period of time.

I also get the impression from Spock's and Ayelborne's comment in Act 4 that the Organians look upon humans, Klingons and the rest as if we were like bacteria or a fungus. They're capable of interacting with us, but they would prefer to avoid getting athlete's foot or diarrhea or whatever. So shoo... go 'way...

And both warring parties obviously knew this right from before the treaty was even written, as Kirk and the Klingons never hesitated to take up arms after that. Clearly, nobody was ever worried about Organians looking over their shoulders.
