Asking this question about Alien is kind of weird. What if Star Wars had been a flop?
Then we would have gotten either a smaller sci-fi boom, or Close Encounters of the Third Kind may have nudged the boom in a different direction (no SW rip-offs, lots of aliens/UFOs and CE3K ripoffis). Probably no ST:TMP - instead we get the Star Trek - Phase II series on TV which, without Spock/Nimoy doesn't last too long and then we don't get the string of Trek movies through the 80's.