I feel there are some noticeable differences. They're not things that good playing can't overcome, but they certainly help. It's the reason why I've avoided India or Egypt for cultural victories. I don't want to get used to getting every wonder or having huge cities with no unhappiness problems. My first game was with Songhai and I noticed that getting only 25 gold with barbarian camps felt sad afterward. I was also confused at how quickly my ancient era units died when at sea.
Overall, their advantages seem to be dependent on play style, but some have more uses than others. Plus, there's UUs to consider. Certainly, using Companion Cavalry is a noticeable difference compared to using regular horsemen.