In addition, I think the way we wrote the Andorians in ST:E, and the way they're portrayed in Journey to Babel (the Ambassador describes his people as a violent race) is that they wouldn't necessarily run to join up with a peace keeping organization like Starfleet, and even if they did the standards of Starfleet Academy might be difficult for them to attain (Kirk says in "The Apple" that Mallory's father helped him get into the Academy - that implies it wasn't easy or a given that anybody could go). But there would be the occasional Andorian who might get in.
The angle I've taken in the post-ENT, early-UFP setting is that the UESPA Starfleet is the exploratory and diplomatic arm of Starfleet and the Andorian Guard is the defensive arm. So they're both in Starfleet, but in different subfleets with different (though occasionally overlapping) responsibilities. I've been assuming they'd eventually evolve into the more unified force we see in the later shows, but I suppose it's possible that the separation could've persisted into the TOS era. There was an IDW TOS comic story (one of the issues of Burden of Knowledge) that portrayed the Andorians and Tellarites as still being fairly separate and mutually unfriendly societies with their own distinct fleets (still using the exact same ship designs from a century before). I found that interpretation kind of iffy, preferring the idea of a more unified Federation, but it's not out of the question.
I've often thought that since we never saw the Intrepid in "The Immunity Syndrome," it's conceivable that it was a ringship of Vulcan design, rather than the Constitution-class vessel it's generally been assumed to be.