They lucked out getting Michael Piller in. He replaced Maurice Hurley, who'd been doing the job for most of seasons one and two. Piller brought a different sensibility to the series to what Hurley had been doing, although Hurley still contributed a script or two during Piller's first run.
The difference in tone is startling. I've seen a fellow TrekBBSer describe it as being like the producers were saying, 'Hey, that stuff we did the first two years, well we're going to start doing it properly now'. It's like TNG was a child who was finally becoming an adult. It's exactly like that. It's as if Piller sat everybody down in the writer's room after he took over in season three and just said to them, "Okay guys, let's make this series grow up a bit now".