That's when the bridge has a soon-to-be-skeleton crew of redshirts.
Or it's when all those guys we only saw only once or twice or a few times (Leslie, Kyle, Riley, Galloway, etc.) take over. Then there's all those episodes where we only saw Chekov because George was working on The Green Beret
, so maybe Sulu was waiting for his shift to start.