Yeah, they're making up the details of the tech as they go along at this point.
One could argue, I suppose, that the ship has special sensor suites to detect and examine FTL spacecraft, different from the devices routinely used to measure mass and velocity etc. of stars and other "natural" celestial objects, and that they've just never been calibrated up to this point for such a huge object. But that would be beyond nerdy.
In the real world, if we wanted to calculate the mass of a spacecraft powered by rocketry - using today's technology - we could analyze the spectrum of its exhaust plume. If you can he how much exhaust there is and how it's moving it's easy to calculate how much thrust is being produced. Combine those observations with a measurement of the change in velocity of the ship itself and it's mass becomes a simple calculation.
I am sure these principles could translate rather readily to different types of propulsion. So, Spock got a reading on what time of propulsion the ship used, or at least the energy emissions, and observed the ship's movements, plugged them into the computer, and got a Ludicrously High Estimate.
/Beyond nerdy