There was also the premise in Voyager that half the trained and qualified Starfleet crew complement had been lost during the catastrophic trip to the Delta quadrant in the pilot episode, necessitating their joining of forces with the Maquis crew and leading to duty shifts by some characters in areas other than that of their specialty (e.g., Tom Paris doing some duty in Sick Bay because he'd received some secondary medical training.) Voyager fudged on this often enough that it was good to see them at least trying to remember that "Oh, hey, we probably don't have a dedicated ship's geologist any more - he was killed in 'Caretaker, Part 1,' right?"I recall one episode of Voyager where the crew had to go on a geology mission. Instead of introducing a ship's geologist, they picked main characters who had taken geology for one semester and Neelix because he went in a cave once.
Budgetwise, of course, they usually need to maximise the use of the contracted feature actors.
They didn't lose as many crewmen as you think but another example is Ensign Wildman, an honest-to-god science officer. Basically they used her once or twice a year as part of the latest mommy in space story. She never got to be involved in any science plot that didn't also involve her role as a mother - that would always be shuffled off to Janeway, Kim, or Seven.