if you want real worldbuilding, then TNG is when that stuff really started.
I think I'd offer a partial disagreement here. The deepening of lore really started with TAS. A surprising number of the episodes were given over to things like filling in Spock's backstory, returning to Mudd, returning to the Shore Leave planet, more tribbles, more Klingons, more Romulans, more Orions, etc.
Obviously, there was more than that. But they were clearly throwing bones to the developing fandom who was doing early theorycrafting.