It failed because VHS, after a few years, boasted 8-hour duration (at the slowest speed, on the longest cassettes). Never mind that the quality at that speed was, if you'll pardon my Yiddish, bupkis.
If 3/4" U-Matic (hardcover-novel-sized cassettes, with a 1-hour maximum capacity) had ever been offered as a consumer format, I'd have been happy to buy it.
But shed no tears for Betamax: with a bit of re-tooling, it spawned the Betacam and Betacam-SP broadcast formats (a cassette the size of a Betamax cassette holds 30 minutes, and in an emergency, you could record [non-SP] on an actual Betamax cassette; a larger cassette, about the size of a 3/4" cassette, introduced in the Betacam-SP format, holds 90 minutes). Of course, all of this became obsolete with the move to HDTV.