In 'Scientific Method' when Janeway is going a little nuts and ranting at Tuvok, she says about how lax things have gotten on the ship and that, as Security Chief, thirteen department heads report to him daily.
Not including himself there are four seen onscreen (Paris, Torres, Kim, EMH). I'd assume there is a senior science officer kicking about somewhere onboard (they just always seem to miss the scheduled meetings), which would bring the number of officers who report him to five.
So what are the other eight departments? The only thing I can think is that services such as the shuttlebay, cargo bays, transporters, and individual science sections (life/planetary/stellar/etc) reported directly to him, although I always see them as being sections of larger departments (flight control oversees the shuttlebay, whilst cargo and transporters both fall under operations).
For a ship with just 150ish crew aboard it seems like quite a lot of departments.