Is there any novel about Janeway, once back on Earth, facing a court martial over the whole Tuvix thing?
No, nor is there any reason to believe there should be. Captains often have to order officers to their death. That's part of their responsibility. Janeway sacrificed one member of her crew to save the lives of two members of her crew. She had an impossibly hard decision to make, one where there was no good outcome and there were strong arguments on both sides, and ultimately it came down to her command judgment, which is a captain's whole damn job. It was ambiguous enough that there was probably some debate, some hard questioning about her reasons for her decision, but I see no evidence that a crime was committed.
The only reason Kirk was court-martialed for Ben Finney's apparent death in "Court Martial" was because the doctored evidence showed him jettisoning the pod
before there was an imminent threat to the lives of his crew. In reality, once that threat became imminent, he was considered justified in making the painful choice to sacrifice one crew member to save the greater number. We've seen captains do that many times, sending officers on likely suicide missions (Sito Jaxa, Spock in "The Immunity Syndrome"), sealing off a breached section with crew inside to save the rest of the crew, or even killing officers who've become threats (Gary Mitchell, the Borg-assimilated crew in
First Contact). That's within a captain's lawful responsibilities when it becomes necessary. If Tuvix had lived, Tuvok and Neelix would have both "died." The threat to their lives was imminent and self-evident. Either way, at least one crew member would have been lost, so it's impossible to say that one choice was clearly wrong and the other right. Janeway chose two people's lives over one, which is a simplistic way to make the choice ("I refuse to let arithmetic decide such questions" -- Jean-Luc Picard, 2364), but the only available basis when all other factors balance out.
