Yeah, it was a good show but at a certain point I don't even think it was *trying* to make sense of the "science" behind what was happening. Hell, it reached a point where it even stopped trying to make the futuristic technology look like something that could exist in the next decade rather than something made out of clear Lego. See: The evolution of Al's comlink culminating into the look of the QL:Project headquarters/base when Sam leaps back to the "present." That place was a compound that might as well have been the Starship Enterprise considering the look of it, how the technology seemed to work and relay information and even the number-dropping of various computer stats.
In the end, though, it does seem that more than not Sam's physical body was leaping (re: the above mentions of times Sam does something his "host" persona wouldn't have been capable of.
But at the same time it does open the problem of how Sam can fit into the clothes of his hosts. And, okay, hand-wave that the "aura" of the host makes Sam *appear* to be that person and sound like them. That doesn't explain how Sam feels when touched. (I.E. When he's a woman how no one notices when touching him he' rough and hairy.) Not to mention how Sam shaves, cuts his hair or, hell, how he stays healthy! What happens if he gets hurt or sick?!
Early-on it seems Al saw Sam as the "host" (see: when he's lusting after Sam as "Samantha" in Sam's first cross-gender leap) but later on (in the final episode, I believe) it's said that Al sees Sam as his "real self."
Think about the show for too long and it really becomes a mess. But, overall, it was a good show.