Depends on how the extra speed was achieved. Perhaps it's just a matter of feeding more power to the warp coils - so once one would yank the portable Kelvan powerplant out of the loop, all the advantages would be lost?
Certainly it seems that the standard way to make the ship go faster than planned is to disengage the failsafes for doing so. It's not as if the drive itself would be incapable of warp 14 or whatever; it's just something that cannot be safely sustained, with the failure points relating to ship's structure (just as in the extreme case of "Threshold"!) but possibly also to the power arrangements.
Good point. It is possible the Kelvans simply diverted all the power from the shields, weapons, and for that matter most of the life support (that is implied when they reduce the crew to the blocks).
Something completely impractical on a regular basis.