Well, seriously speaking, Spock never
really uses fractions of a second. He notoriously uses two decimal places for fractions of a minute in "That Which Survives", but in fact with some justification, as there is a very accurate clock ticking towards ultimate doom, and in the end it's seconds that count when Scotty saves the day.
Okay, so the difference between 11.5 h and 11.337 h earlier on in the episode is over a less crucial bit, a mere ETA. But it does amount to almost ten minutes, even if it isn't the most illustrative way of putting it. It's mostly a case of Spock not fully understanding the nature of human units, and when to use decimal points vs. sub-units.
Timo Saloniemi