I thought the explanation Cushman had in his book made a lot of sense. That between having to spend all his time learning lines, and the stress of his marriage and home life falling apart, he just didn't have the time or motivation to focus on exercising and keeping the weight off as much.
Well, that and he clearly likes to eat.
Actually when it comes to the third season, I thought what was more
unflattering with his appearance was the shaggier 60s hair and the flat looking toupee he wore that lacked the blonde highlights from before.