In the relevant VOY episode, "Fury", we get a very specific context for the rhyme about left and right.
Janeway: "Tom, what's the first thing they teach you about manoeuvring at warp?"
Paris: "Faster than light, no left or right. When possible, maintain a linear trajectory. Course corrections could fracture the hull."
That's
the first thing they teach. That is, a general starting point. A silly rhyme that is meant to be remembered "when possible".
It's no different from saying that fighter jets should fly straight. That's very true: g-forces are hard on the structures, and a jet that maneuvers will soon be an ex-jet. But that is just a general starting point, because in order to use the jets for something worthwhile, they
have to maneuver, even if it is in theory bad for them.
To be sure, the situation in "Fury" called for truly extreme maneuvering. The ship was stuck in an area of space Swiss-cheesed by "subspace vacuoles", much as in TNG "In Theory". Except it was worse: 70% of the volume was untraversable. This was not a case of gently easing the ship into a 30 degree turn along a distance of a few million miles. They were turning sharp corners on the scale of
kilometers.
Nothing in "Fury" is inconsistent with the rest of Trek in this sense, then. Kirk's ship could pivot at warp 2 ("Elaan of Troyius") but probably couldn't follow
Voyager through those twists and turns at any warp speed. Probably no ship could, as
Voyager was specifically famed for her maneuverability.
Anyway, that pivoting thing from "Elaan of Troyius" is an interesting one. Can a ship at warp fly in any orientation other than nose first? The dialogue in that ep suggests the answer is yes. And in the opening credits, we see the ship fly at supposed high warp nose decidedly down, "forehead first". Kirk also reversed at emergency warp in "Balance of Terror".
So the real question only seems to be, can all starships fly at warp in arbitrary orientation, or are there forbidden orientations for certain ship types?
Timo Saloniemi