In this case, the consequences of NOT having the conversation are what you should be afraid of.
I agree, because it's usually
not talking that degrades and destroys things over time. Especially if you're unhappy with something.
Especially if you think not talking will preserve the relationship.
Thirded. In mono relationships, good communication is the difference between a good relationship and a bad one. With poly, good communication is the difference between a good relationship and complete fucking disaster. I'd need to talk with you more before I can offer any input about how to change this.
Also, it might be a good idea to go to
polyamory.com and read some threads. Do a tag search for "mono/poly" or something. Or post a thread. The people there will probably point out some relevant threads and/or respond directly. I post over there too, under a different name, so I can talk about people without them finding out.