Would one say, "In all the years I've known you" if it'd been fewer than, say, seven to ten? It's reasonable to infer from "all" in that context that Kirk isn't just speaking about the five-year mission he's commanding.
Kirk addresses Pike rather familiarly, as "Chris," in "The Menagerie." Generally, one doesn't do that with a superior if you know him only casually; it's considered a rather gauche breach of protocol and propriety.
Here's a question that arises on occasion: Could he have served as Pike's XO for a time before taking command of Enterprise? I ask because I don't recall if there's anything in the canon concerning his assignments pre-Enterprise that precludes it.
A few months as the heir apparent being groomed by an experienced commander would seem a minimal requirement for the center seat on a Federation starship, no matter how hot a shot Kirk was.
There's nothing really saying Kirk didn't serve on the Enterprise before being Captain.