I actually made this exact point five (six? seven?) pages ago. It works very well for TOS at least, where you ca conclude that all the guys in the yellow shirts are military while the reds are non-military government agencies and the blues are from NGOs or science specialists. It doesn't work as nicely for TNG since they alot of officers switch colors when they change specialties or get promoted, and because the red/command colors of TNG don't appear to be focussed on the military aspect (the yellow shirts do, because they do most of the combat and tactical planning, e.g. Yar and Worf). So there's a lot of overlap in the spinoffs that makes that differentiation tricky.