From Before and After
PARIS: That was the beginning of the year of hell.
KES: Year of hell?
PARIS: That's what some of us call it now. We were under virtually constant attack by a race called the Krenim. The ship almost didn't make it. The Doctor was offline for months, and we lost a lot of good people. Captain Janeway, Joe Carey, B'Elanna.
PARIS: I keep forgetting that you don't remember any of them. B'Elanna was someone who was very special to me. When she died, I felt like I wanted to die too, but you, you helped me through it. I'm sorry.
KES: No, no, it's my fault. You haven't done anything wrong. It's just
Other than the TNG episode with Josh Clark in it which some bright sparks insist is Joe Carey, above is the only mention of Carey in an episode which Joe didn't appear in.
So, it seems like he got 3 extra years than he was supposed to.