I understand that because of the assumption of a second five year mission before TMP, the version of the Enterprise and the design of the uniforms doesn't always match the era of the book with regards to some older TOS novels.
No, there's no cause and effect there. There was never any formal, widespread policy of assuming a second 5YM; it's just something that a few of the novels implicitly seemed to be assuming (like
My Enemy, My Ally, which was supposed to be seven years after "The Enterprise Incident" but still had Sulu as a lieutenant and Chekov as navigator). We're so used to having a clearly defined chronology for Trek today, but until 1988, there had never been a calendar date established for anything in present-day Trek. True, in TMP, it was made pretty clear in dialogue that Kirk had only commanded the
Enterprise for five years, but in the days before home video and the Web, it was easy to forget details like that. And some people evidently assumed that since TMP was made a decade after TOS, it must've been set a decade after TOS too. Nobody ever formally came out and said so, but nobody really said otherwise either. The chronology was so ambiguous that different authors made different assumptions and weren't always on the same page. And it's just in retrospect that we look back on that part of Trek Lit history and deduce that a couple of authors seemed to be assuming something different about Trek chronology than most people did.
The mismatches between cover and content are for reasons totally unrelated to any of that. At first, since the books were coming out in the wake of TMP, it made sense to use TMP designs on the covers for promotional purposes, regardless of the books' content. The same way that, say, a reissue of a classic novel that's just had a movie adaptation made will stick the logo and star of the movie on the cover even if the movie has drastically changed the storyline. It's all about cross-promotion.
Later on, it was probably just a matter of different cover artists drawing on different photo references or having different preferences.
But what about something like The Better Man by Howard Weinstien, which was obviously written to be post-TMP (stardate and references to the film), but which sports a cover with TOS-era Enterprise, Kirk, and Spock? Editorial boo-boo?
Cover art for tie-in novels is often done before the manuscripts are finished. A cover artist may only have the outline to go from, or an editor's instructions based on the outline. Really, most novels that are set post-TMP haven't taken much advantage of that setting, haven't been too different from a 5-year-mission novel, so it can be hard to tell from the outline just when they're set.
Really, though, I think most people buying books are looking at the faces, not the uniforms. Whether the details on the cover match the story is a secondary consideration. The goal is to create an image that's appealing to the eye and makes the observer curious enough to pick up the book and read the blurb or page through it.
Doesn't the majority of The Better Man flash back to events set during the original five-year mission and/or before?
No. There are assorted scenes that flash back to events in McCoy's life ranging from 2236 to 2265, but they're nowhere near the majority of the book.
(And hey, I just noticed there's a starship in the 2250s scenes with a crewmember named Kara Choudhury. Maybe an ancestor of Jasminder...?)