I was thinking about this topic generally and specifically related to the definition of sci-fi versus fantasy and for me
one of the key elements of fantasy that differentiates sci-fi from fantasy is in fantasy is the belief or element of the supernatural whereas generally speaking in sci-fi the supernatural is disregarded or even mocked.
For example: in Trek in TNG in the episode Who's Watching the Watchers
Picard mocks the Mintankans belief in "The Picard," as a "God," and implies that a belief system in a deity is backward thinking.
DS9 furthers this notion with the Profits. The 'advanced federation folks,' refer to the Profits as 'wormhole aliens,' and look down generally upon the Bajorins worshiping them as gods.
The Q while it could be argued are more 'mystical,' are always presented IMO as simply a very advanced species. Their 'magic,' isn't magic at all but simply a product of advanced evolution.
Therefore, Trek dismisses the mystical and supernatural generally as nonsense and attempts to explain things through the prism of science.
Conversely, films like Avatar
and more hard fantasy like Harry Potter
embrace the notion of the supernatural. The characters don't tend to debunk the supernatural but latch hold of it. In Avatar's
case, the goddess of the forest is eventually found to exist and is supernatural even by the visiting humans.
Back to the original question - why is fantasy more popular than sci-fi generally?
I believe that most people on this planet want to believe in the existence of a god. 90% of Americans in fact believe in the existence of God
. Fantasy and the belief in the supernatural tends to support their belief system whereas sci-fi from their perspective seems less probable despite the fact that it's more based on scientific facts.
You only need to go to the number of people who believe in intelligent design or outright creationism to come to the conclusion as to why people are more likely to embrace fantasy over science fiction.