"The Cage" and "The Corbomite Maneuver" both make a point of the captain being weirded out by having a female
yeoman. As if that was far from the norm.
But then for the entire rest of the series, yeomen are always female, at least the ones addressed as such. I would even venture that many
Star Trek fans grew up thinking yeoman was a feminine job category
per se, the way nursing, secretarial work, and teaching elementary school were for decades.
The thing is, the original plan was for the captain's yeoman to be a regular character. In the early first season, Grace Lee Whitney was billed just after DeForest Kelley in the end credits; they were lumped together under "Featuring," making them both the second tier of regulars after Shatner and Nimoy and above everyone else. So the original plan was that, yes, a female yeoman would be unusual, but we'd only see the one for the most part. Once Whitney was dropped from the show, though, instead of replacing her with a new regular, they just tossed in a different guest yeoman every week. Initially they rewrote Rand's part in the scripts for other yeomen (like Mears in "The Galileo Seven" and Barrows in "Shore Leave"), so they were still female. Maybe they tried out various guest stars in hopes of finding one they could promote to regular status, but then they just got into the habit of using a different yeoman each week.
And they just ignored the early episodes' lines about female yeomen being rare, because it was the '60s and continuity didn't matter much because nobody expected people to see the episodes more than once or twice, or spend decades studying past episodes in exhaustive detail and dwelling on every inconsistency.
Was there some storytelling advantage to making the yeoman of the week female? Or was it just a (civilian) cultural expectation that someone doing what amounts to secretarial work ought to be a woman?
The main advantage was sex appeal, to draw in male viewers. They wanted to have pretty girls on the screen, and by the standards of the time, the most believable/acceptable roles for women in the workforce were as secretaries, nurses, waitresses, and the like.
I remember being a little miffed when the Captain was annoyed with having a female yeoman. It felt a bit strange that one of the most prolific captains of Starfleet, an organization who purportedly claimed humanity had achieved utter equality, would even care what sex his yeomen were.
I see it as being aimed at the viewers. Often, if you know that your audience is likely to be uneasy or skeptical about something in your story, you have a character in the story voice that same attitude to get the audience on their side, and then show them learning to accept the thing. So I think that Pike's and Kirk's initial discomfort with their female yeomen was meant to briefly acknowledge the concern that some audience members would have with the idea, and then just move beyond it and have Pike or Kirk get over it, hopefully bringing the audience with him. No, it didn't make a lot of sense in-universe, but it served a purpose metatextually.
On the subject of women in Starfleet, I'm pretty sure we never saw a female security officer.
In TOS, no, but in the animated series we saw Lt. Anne Nored in "The Survivor" and an all-female security team in "The Lorelei Signal." There were some female personnel in security colors in the rec-room crowd scene in ST:TMP as well. I don't remember if there were any in the later TOS movies.