True enough..
Mind you, some ppl don't regard the Brig as a full fleged companion, and he's been in multiple seasons.
Has the Brig ever actually travelled in the TARDIS?
* * *
My personal sense would be that a character qualifies as a companion if they willingly subordinated themselves to the Doctor in the course of an adventure, aiding the Doctor in resolving a crisis. So, to my mind, for instance, Captain Adele Brook in "The Waters of Mars" was not a companion, but rather an ally, because she never relinquished her command of the station or subordinated herself to the Doctor. I'd say the same about Harriet Jones, Prime Minister, in "The Christmas Invasion."
By contrast, I'd consider Astrid and Lady Christina both to be companions, even though neither one travelled in the TARDIS and Adele did, because both willingly subordinated themselves to the Doctor in the course of resolving a crisis.
So in retrospect, I suppose the question is not, Did the Brig travel in the TARDIS, but, Did the Brig subordinate himself to the Doctor's authority?