Perhaps the Constellations were fast transports of some kind, maybe for troops.
Connies look to be more long-range exploration/science ships. A major secondary hull obviously isn't needed for warp travel, but the connies use a lot of space for that. Would free up the saucer for labs, sensors, and crew quarters, and a bunch of room for cargo necessary for long tours. The single-hulled ships might not be so long-ranged - but they also wouldn't need as much major maintenance, and would get more frequent tune-ups and the like, so they would last longer.
Of course, there could still be Connies and Constellations about, we just haven't seen them. Star Trek Online still has them, and that's set 40-50 years after TNG. There are, however, more modern-looking variants on both (and the Miranda, too), so, at least in that game, each has a special niche to occupy, whatever that is (the Miranda is basically cannon-fodder, like in DS9). (There aren't a ton flying around, as they're all low- to low-mid-level ships.)