Speaking strictly for myself the series starts to fade after Season 3. No slam against Nimoy, but the absence of Martin Landau and Barbara Bain is felt. The series soldiers on, but it's not quite the same anymore. Landau was fantastic and often managed to make so much more of the characters he portrayed.Like a cheap knockoff of Rollin Hand without the star power.
In Season 5 (1970-71, Nimoy's last season) they make changes to the show and not for the better I'm afraid. The '70s look starts to become apparent and most particularly in the soundtrack. Gone is the distinctive and iconic M:I mission cues replaced by something godawful meant to sound more street. Yuch! The stories now started to fucus more on busting criminals than international espionage and such. It was becoming a variation of a cop show.
When Barbara Bain left they should have replaced her with Lee Merriwether (who definitely wanted the role) as a regular. The female characters that followed Bain and Merriwether's occasional appearance were cyphers and totally forgettable.