Um...I don't know if this has already been suggested earlier in this thread, but it is not the plausibility or even the possibility of the imaginary science or technology in the work of science fiction that differentiates it from most fantasy. What makes it science fiction rather than fantasy is whether, inside the fictional world, those fictional characters consider it a form of science.
Science and magic are NOT the same thing. Even implausible or impossible science is not the same thing as magic. For example, magic, to function, often relies on the personal skill, power, concentration, etc, of the user. Scientific experiments, however, once understood, and with a good, clear instruction manual, can be reproduced, more or less, by any shmoe off the street. Science is provable and repeatable, given all the same variables. It is knowledge that can be passed down and used by anyone who desires to. But no matter how much Frodo memorizes the right words, and states them in the right tone, he'll never be able to scream, "THOU SHALT NOT PASS!" and make crazy shit happen. Only Gandalf can do that. It's not repeatable. That's one of the reasons it's magic, and not science, and why LOTR is fantasy and not science fiction.
Warp drive may be as impossible as Gandalf's spells in our world, but inside the fictional world, that world conceives of warp drives as science. Anyone who studies warp drive long enough can work one. Warp engines work the same way, no matter who's controlling them, and given the same variables, they will always work the same way. So, it's impossible (or fictional, rather) science, rather than magic, or fantasy.
This difference is not, to my mind, superficial or merely set-dressing. It suggests something very different about the way that fictional world works, and the relationship between those fictional characters and their fictional environment. DO they, in fact, live in a world that is potentially understandable and controllable by the masses, given the opportunity? Or do they live in a chaotic world of magic, in which only a certain elite few will ever have control, and, in any case, nothing is predictable, provable, or repeatable anyway? Those are very different fictional worlds, and it changes everything about the story, or at least the good ones.