Several quick observations:
1) For someone into polyamory, you seem to have difficulties relating to people. Perhaps polyamory is a cover for this because you don't really have to bond with one person? I bet you are polyamorous in situations that are designed to promote polyamory so you don't have to do anything other than show up to get some.
2) It's just a fact of life that you'll have to compete for most any girl. They don't live in isolation and will have pre-existing relationships. Don't be surprised by this. Happens all the time yet new relationships can and do form.
3) The reason for your "mysterious anxiety" is fairly obvious, it's requiring more interpersonal skills than you are used to. This eye contact communication is new to you. It's not the overt, set-up situation that you are used to. The anxiety is you just have to stretch to be able to interpret and work with this novel situation.
4) You seem fairly negative about many interactions. This isn't helping you!