New canon that conflicted with the direction taken by trek lit wouldn't affect my appreciation of the novels. I became a Star Trek fan around 2001/2002, and the current novel line has always been just as integral to my enjoyment of the franchise as the canon material. I'd likely relegate any new film contradicting the direction those novels have taken to a "Myriad Universes"-type scenario. In other words, I'd personally shunt the new canon into the hypothetical, even if it's supposed to work the other way 'round
No matter how good a new film, I doubt it could measure up to the enjoyment I've gotten from the many dozens of novels; disregarding such an engaging and complex take on Trek in order to incorporate a small slice of on-screen goodness isn't something I'd be willing to do. Given my preference for fictional worlds that are self-consistant, I'd have to "favor" one over the other to retain my full enjoyment; that will seem foolish and unfortunate to some, I know, but it's how I work.
It doesn't mean I'd be automatically displeased with a new release.
As for what would happen to the novels, I'd hope that the current line and its ongoing arcs are popular enough to survive on their own merit despite not agreeing with a new film's interpretation. Even if it all has to be declared "even more hypothetical than it already is", what would be the point in abandoning a take on Star Trek that's been going strong for a decade, just because a new take has arrived on the scene? The imperative that novels must agree with canon is well and good, but I hope the powers that be would acknowledge that the situation is different nowadays - the novels outlasted the TV series they were based on and have constructed a very intricate and expansive world that would lose much of its appeal if it had to be dragged back onto the rails just because a new, short stretch of track has been introduced.
Yes, I know this sort of happened once before, when TNG aired and the budding novel continuity of the time was "invalidated", but I can't help but think of that as a rather different situation. TNG was a kind of fresh start for Trek, at first deliberately distanced from TOS. Any new TNG-inspired film wouldn't represent a grand new vision for success, it would just be another spin-off expansion of the TNG universe. Back when regular Trek was airing, I assume the reasoning behind tight conformity to canon (not just in background details of the universe but in how characters were represented, etc) was that a reader might be alienated if the books they picked up weren't extremly familiar to them and conformed closely to what they were seeing on TV (why would they pick it up if they didn't want to experience a slice of familiar Trek?). But readers now have been either following or occasionally dipping into a novel line that's been successful for almost a decade without an accompanying TV series. And Trek has expanded into so many different forms with fresh interpretations - Star Trek Online, Abramsverse, etc. Surely the novels would be granted a bit of freedom to do their own thing now?