Unfortunately, not all USB ports are oriented in the same direction. Pretty sure my home computer's ports are opposite from my work computer's. Then there's the ones sticking out from the sides of my monitors.....Yeah...I know it's probably just confirmation bias, but it really does seem that I try the wrong side every fucking time. I do the exact same thing when I try to connect my insulin infusion set to my body.
Maybe there is actually something about a USB design that does cause people to attempt to put it in the wrong way up more often than not? Some quality that is on the bottom of the USB that we usually associate with being on the top of things.
A paint pen is your friend. A dot of silver paint or any color that contrasts will make your life less 1st world. I also mark what any connection device is used for.
Edit: Colored paint pens are your friend. Dot match the male and female end!