Edit: Plus, a branching multiverse wouldn't be infinite anyway. It would be the set of all possible configurations of the observable universe that could be reached from some possible path of branches starting at t=0. That's huge, but as the observable universe is finite, so's the number of possible paths. (And since anything outside the observable universe by definition couldn't have any impact on anything within the observable universe (or else it would itself be observable and thus part of the observable universe), you don't even need to worry about the question of the finititude of the universe as a whole.)
Right. People like to invoke the "infinite multiverse" line as an excuse for any random nonsense, as if that somehow made it scientific, but in actual scientific practice, if your equation turns out an infinite result, that's usually evidence that you've done the math wrong.
In practice, if we're talking about divergent timelines, we're not even really talking about the whole observable universe -- just about the ensemble of particles that have been able to interact with each other in the time since the diverging incident occurred. So in a physically realistic universe, that would mean a sphere n light-years in radius for a divergence that happened n years before -- although the sphere would be rather larger in a universe with FTL phenomena like subspace and tachyons and stuff. Either way, though, it's an even smaller finite number of particles, and the set of possible configurations would be commensurately smaller as well.