The show spoke of "Duranium" being used for the hull, or as part of the hull. If duranium refers to depleted uranium (with their science completely depleted) that would make the outer shell of the ship responsible for a health percentage of the ship's mass. Combine that with the warp coils, the impulse engines, and things like the M/AM reactor and the majority of the ship might not be unusually "dense."
I figure duranium is actually the trade name for an alloy that uses a combination of rare metals (titanium, nickel, platinum etc) that are only plentiful in asteroids. Nickel-platinum alloys, for example, have shape-memory properties that allow them to deform hugely under mechanical stress only to return to their original shape when heated beyond a threshhold point; if Starfleet is using something like this in their hull plating, then "polarizing" it with an electric field might be a sneaky way of forcing those hull materials to constantly return to their original shape when subjected to mechanical stress. Better yet, an open-cell metal foam of nickel-platinum alloy would give you a material 90% lighter than solid metal while retaining almost all of its strength in the same volume, plus the benefit of being better able to deflect projectiles in hyper-velocity collisions (rather than shatter/spall into the compartment).
Even better, an alloy of nickel-titanium-palladium would have a mindblowing strength-to-weight ratio in addition to being able to trap some hydrogen in the matrix (palladium hydride) which would make it an ideal radiation shielding material. The same material in an open-cell foam could be fashioned into hull plates several feet thick that are still no heavier than ordinary sheetmetal.
Besides, we know that Starfleet uses a lot of tritanium in their construction too, which I can only assume is an allotrope of titanium (probably a glass).