With the proviso that I object to the term "literary" in general, as I think it denotes a level of snobbishness directed at perfectly good stories as somehow being untrue literature...
The most sophisticated, best-written
Star Trek novels I've ever read would probably be:
- The Final Reflection by John M. Ford
- DS9: Hollow Men by Una McCormack
- DS9: The Never-Ending Sacrifice by Una McCormack
- Terok Nor: Day of the Vipers by James Swallow
- Typhon Pact: Rough Beasts of Empire by David R. George III
- Typhon Pact: Plague of Nights by David R. George III
- Typhon Pact: Raise the Dawn by David R. George III
- Typhon Pact: Zero Sum Game by David Mack
- Destiny by David Mack
- Mirror Universe: The Sorrows of Empire by David Mack
- S.C.E.: Wildfire by David Mack
- Articles of the Federation by Keith R.A. DeCandido
- Spock's World by Diane Duane