The whole reason Vertigo started was to provide space for the niche comic
No it wasn't. It was a part of DC editorial lead by Karen Berger that decided they didn't want to play in the DC sandbox and took many of the horror related characters with them.
--now all the titles have been mainstreamed and have to duke it out with the heavy hitters
As opposed to under Vertigo where nobody did anything with Animal Man after his first book ended and where you had two mediocre volumes of ST after it was cancelled under Millar?
It's truly amazing how readers wax nostalgically for Vertigo when they did a disservice for many of these characters.