I'd say she IS sure that Jellico isn't convinced that what he's doing will work... because he isn't. What she's sensing is probably true. He admits as much to Picard, privately, that he expects their missions to both fail, in the worst way.
But how Deanna framed what she said, was that Jellico is unsure of himself, which isn't the same thing. To be unsure of himself would mean he questions himself as to the choices he's making, vs some other choices he could be making.
She's suggesting that he's undecided about his own actions, which I'm convinced isn't true. He came into this with a specific plan of action, based on profound experience in similar situations with Cardassians. He's moving forward in the best way he sees fit, to ensure the highest probability of success. He's even bucking against the crew some to see he gets it.
My point is that we get no other indication that he is hiding self-doubt. He seems almost single minded about what he must do. However, he likely suspects that even doing everything right, like he's trying to do, the mission will still fail
That's the doubt he's feeling, & I'm not convince Troi is finely tuned enough to differentiate between someone experiencing self-doubt, vs just ordinary doubt, like about the mission, and while she was probably just making an off the cuff remark, it suggested the man is afflicted with self-doubt
Picard's rarely ever been afflicted with self-doubt about himself, but he has been similarly full of doubt about other stuff plenty of times, & she's never made such a comment about him, because she trusts him. The one time he was full of self-doubt, (Time Squared) she nearly had a meltdown trying to tell everybody.
It was very careless to drop such a thoughtless remark, when people might misconstrue it, especially when I suspect she's not even capable of making such a delicate emotional distinction
She sensed doubt. So what? What kind of doubt is what matters, & I really doubt she could ever be that specific, but her comment was very specific
