I don't think it would have been very appropriate for Bond to be 'having fun' in QoS, given his emotional state at the time.
That's why it was a terrible decision to have it as a direct sequel.
You know there is precedent, right? "On her Majesty's Secret Service" and the following "Diamonds are Forever". Now, I'm not saying they should have gone all-out wacky the way they did with "Diamonds", but some balance would have been nice. Also, the "too much" of the wackyness of "Diamonds" didn't really come from Bond's character, but from the other characters.