Yes he was. See the ending of "Nightmare in Silver" where he catches himself in a reverie about her tight dress. And he was way more touchy-feely with Clara than Amy. Also describing her as "perfect" in the "Name of the Doctor" prequel minisode (in that same minisode, Clara admits to falling in love with him).Matt Smith's Doctor was obsessed with Clara because he couldn't explain or understand very existance. I don't think he was romantically attracted to her though.
I mean, of course he was obsessed with her in the sense that she was a mystery he felt compelled to solve, but there was attraction there on both sides. The writers made it as overt as they could without it descending into Ten/Rose territory.
Also, in "Deep Breath", he says: "Clara I'm not your boyfriend."
Her: "I never thought you were." (lie)
Him: "I never said it was your mistake."
And really, Clara was a good fit for Eleven. They did "work" as a couple. Moreso than that nonsense with River Song.
I do wonder if, when Clara exits probably at the end of series 9, will I view her more as Eleven's companion or Twelve's? At that point she'll have 2 series with Twelve, but she'll probably always fit with Eleven in my mind.
I'm saying that now, anyway. It could change.