As stated above.
At which point the complaints become that they don't try hard enough to have them be in more episodes. Even if it's reasonable they couldn't be in every episode.
That Voyager is smaller than Enterprise is irrelevant; it's still a huge ship. The complaint is that we should have seen more of the background characters as well as learnt more about them.
We should have seen them more often (but not every episode). Just going about their business. And characters that we did see quite a lot - like Ayala - should have been given a few more lines.
Enteprise didnt need to do this because it was just a Starfleet ship doing its job whereas Voyager was supposed to be a family lost in space. It should have attempted to demonstrate this more (by utilising the background characters).