It would make sense if Tarkin really did consider Vader a friend, assuming he knew Vader's true identity. In The Clone Wars, Anakin and Tarkin met on a mission and established a rapport pretty quickly.
That they did. (It was cool seeing Tarkin in that episode)

I submit that it's also possible that Vader was re-introduced to Tarkin during the Moff's/Grand Moff's time at Imperial Court, albeit without Tarkin realizing that Vader was formerly Anakin Skywalker. Tarkin could well have been one of the few, if only, non-Force users let in to Vader's "inner circle" because the former Anakin recognized and knew Tarkin as a ruthlessly efficient officer.
Largely, it was only a few surviving Jedi (and, of course, the Emperor) who would know that Vader was formerly Anakin Skywalker. It isn't really known if Tarkin knew of Vader's origins.
It could be implied that Tarkin did know because of his line: "The Jedi are extinct; their fire has gone out of the universe. You, my friend, are all that's left of their religion."
Implication 1: Tarkin knew Vader was Anakin, and formerly a Jedi.
Implication 2: Tarkin used the term Jedi as a catchall phrase for Force-users much as one of us might use Coke as a catchall phrase for sodas. Hence "you are all that's left of their religion." He might not really differentiate between Jedi and Sith. He makes no mention of Sith in the film.
Implication 3: Same as above, except that Tarkin has no idea that Vader was formerly his old comrade, Skywalker. The "my friend" part being delivered loosely, but not entirely insincerely.
Just my takes on it.
