Ah, yes. "You picked a great time to tell me." Not the most eloquent of lovers, is he, our Tom. BUT. Given how hard he's pursued her throughout Season 3, and now he's finally got to where he wants to be (his attraction to B'Elanna is quite obvious throughout the preceding exchanges -- "... there wouldn't have been a time when I wouldn't have found you fascinating...") I actually thought that line came from the deepest of disappointments, and was quite fitting.
I also found his interactions with B'Elanna throughout quite honest and believable -- he's trying to do something with her, and she thwacks him back just once too many. And still, he's quite understanding with her when she complains about her shit day in the shuttle. Maybe you need to see it again ...?
But then again I'm a hopeless romantic, and an inveterate P/T shipper (full disclosure here now that the spoiler effect is off).