Agreed he was ready to sweep something under the rug, but I wouldn't assume that he thought Kirk had intentionally killed Finney. The charge he brought was negligence.
KIRK: Finney knew he had a matter of seconds. I gave him those seconds and more. But apparently it wasn't enough.
STONE: Then why, Captain, does the computer log from your ship, made automatically at the time, indicate that you were still on Yellow Alert when you jettisoned and not on Red?
KIRK: I don't know. There's been a mistake.
STONE: It would seem so. Could the computer be wrong?
KIRK: Mister Spock is running a survey right now, but the odds are next to impossible.
STONE: Stop recording. Now, look, Jim. Not one man in a million could do what you and I have done. Command a starship. A hundred decisions a day, hundreds of lives staked on you making every one of them right. You're played out, Jim. Exhausted.
KIRK: Is that the way you see it?
STONE: That's the way my report'll read if you co-operate.
KIRK: A physical breakdown. Possibly even mental collapse.
STONE: Possibly.
KIRK: I'd be admitting a man died because
STONE: Admit nothing. Say nothing. Let me bury the matter here and now. No starship captain has ever stood trial before, and I don't want you to be the first.
KIRK:
But if what you suspect is true, then I'm guilty. I should be punished.
STONE:
I'm thinking of the service. I won't have it smeared.
KIRK:
By what, Commodore Stone?
STONE:
All right. By an evident perjurer who's either covering his bad judgment, his cowardice, or -
KIRK: That's as far as you go, sir. I'm telling you I was there on the Bridge. I know what happened. I know what I did.
STONE: It's in the transcript, and computer transcripts don't lie. I'm telling you, Captain, either you accept a permanent ground assignment, or the whole disciplinary weight of Starfleet command is going to light right on your neck.
KIRK: So that's the way we do it now? Sweep it under the rug, and me along with it? Not on your life. I intend to fight.
STONE: Then you draw a general court.
KIRK: Draw it? I demand it. And right now, Commodore Stone. Right now.
Stone clearly leaves open the possibility that it was deliberate, and is still willing to make it go away 'for the good of the service'. And he clearly is writing a narrative of 'mental collapse' not on the basis of any facts but on the basis of what he prefers to have happen.
Also, even if it is 'negligence' or 'mental collapse', it's still corruption to try to hide that rather than see justice done according to the appropriate laws and regulations.