I think this is part of my issue. Who says we would already know about the phenomenon that leads to psi because we would have it on Earth? Seems a little arrogant to think we already everything there is to know. Who says there's not some phenonmena we are completely ignorant of right here on Earth?
That's totally missing the point. I am so sick of people making the mistake of thinking that science is about assuming we already know everything. The whole reason we need science is that we don't know everything. The very purpose of science is to discover new things! Science is the only school of thought that is purposely designed to lead people to new ideas and make them question their old ones, as opposed to religion and politics, which are usually about reinforcing existing beliefs and discouraging attempts to question them.
But your last sentence is the answer to your question. Of course I'm not saying there aren't things we don't know about. What I've said explicitly and repeatedly is that anything that exists elsewhere is going to exist on Earth too. What I'm responding to is the specific assumption that "It's not on Earth" is a sufficient justification to assume that different rules apply. A lot of sci-fi treats outer space as if it were a magical fairyland where any random nonsense could happen, and assumes that just being alien is enough of an excuse for effectively supernatural powers. I'm just pointing out that it isn't as simple as that. If you want to offer a convincing justification for psionic powers, it needs to be grounded in something different from the mistaken belief that the rules are different in space than they are on Earth. Because every time we've believed that in the past, we've turned out to be wrong.
The thinking seems to be "starting with what we know right now, extrapolate to what could be in the future, but if you stray from that, I will obect and call it magic"
That's a complete misreading of my position. I'm open to a novel idea if someone could actually offer one. That's what I've been saying all along -- that most sci-fi invokes the word "telepathy" as an excuse to avoid offering any details or specifics, new or otherwise. It's just used as a black box that we're supposed to accept without question. As I've already said, I'd welcome a story that actually proposed a mechanism for how psi powers worked, but I don't know if I've ever read one or seen one (aside from the one I came up with myself decades ago for a fiction project that never went anywhere).
And calling something magic is not an "objection." I'll never understand our culture's kneejerk assumption that every discussion of difference has to have a value judgment attached. Saying that two things are different does not require saying that one is "better" than the other. It's simply a matter of understanding them. What I'm talking about is simply a difference between two types of storytelling -- the kind that involves working within a clear and consistent set of rules, and the kind that involves allowing anything to happen with no more explanation than a handwave. The majority of stories that use magic use it in the latter way, to allow anything to happen no matter how little sense it makes (although there are a number of fantasy authors whose schools of magic have clear and logical rules that must be followed, and those are my favorite kinds). What I'm saying is that most fiction that uses psi powers uses them the same way. I'm discussing different approaches to storytelling, not making a value judgment.
whereas I am willing to say that maybe "what we know right now" is too restrictive because who says we know even the smallest fraction of how the universe works right now...
We know enough to know that the physics we don't yet know about coexists with the physics we do know, rather than suspending or negating it. When we discover whole new branches of physics, as we've done with relativity and quantum mechanics, it doesn't undo the results we got before, just expands on them. Classical physics is a subset of modern physics, the way things behave under everyday Earth conditions. Relativity shows us what happens at velocities and accelerations far beyond those conditions, but the solutions of relativity under everyday conditions are equivalent to classical physics. Quantum physics shows us how matter behaves on the level of individual particles, but the behavior of quantum particles in macroscopic ensembles averages out to classical physics. So new physics is not an excuse to throw out the existing rule books. It just fills in the gaps, the extreme and unusual circumstances that we don't already have a good handle on.
, so I'm willing to accept more because I think you could make up a scientific explanation for anything even magic.
Okay, then give me one. I'd be interested in that. I like clever explanations for things, plausible or not. As I've been saying, what frustrates me about the usual approach to telepathy is that nobody offers any explanations. They just say "It's psi" as if that were the whole answer. It feels like a copout. So please, by all means, if you can offer a creative explanation for how psi or magic works, I'd be delighted to hear it.
And why do we need to know in such detail how psi works?
Because it would be an interesting and refreshing change of pace. Because hardly anyone has ever tried to do it, and those are the things most worth doing.
Do we know in that much detail how a transporter works?
Actually we do. See Chapter 9 of the TNG Technical Manual. There's also the real-world theory of quantum teleportation, which is more about sending information than matter, but makes a pretty nice hard-SF justification for stories about teleportation.