ENT had a horizontal layout, but it couldn't eject the warp core AFAIK.
Enterprise D and up had vertical along with most others of that era.
I'm pretty sure it's to make the ejection hole in the starship's hull as small as possible.
Since you have to have a physical hull, that ejection port will be considered a weak point since there will be less hull then the other areas of the ship.
Might as well design it to be as small a surface area as possible.