I've long held the suspicion that had Lazenby done more films, he'd be considered the best Bond. Connery really did become too big for the role. I don't think Lazenby would've had that problem. And his character was so much more believable. Plus, as you said, he didn't have the cartoon image of his successors.
As far as the next one, I think some time off will probably be a good thing.