My personal belief is that Janeway wants her freedom.
Jeri had a lot of Mark in her Mosaic/Pathway's, a childhood friend who upgraded into lover at the 11th hour before Caretaker... I say "11th Hour" but how long had Kathryn been actually engaged and was she really in any rush to set a date and would it have changed anything in how she lived her life if she had tied the knot?
It's possible hat he was just her beard. He was there a million light years away so that she wouldn't have to have all that sex with all those aliens like Kirk did. About as likely as that he wouldn't agree to look after her dogs until she took that ring...
Did she wear her engagement ring for the first few seasons and how soon after she found out that he had remarried did she space it?
If that was his Grandmothers ring it was inconceivable of Kathryn not to have given that to Telek R'Mor to forward on to the man she thought of as minutely more than a friend.
Teacake looks at her hands a lot.
Was it a uniform violation?