I've been trying to stress that point for years.
But lets just remember one thing.
He sent the Cardassias back.
But Neelix said that there's dozens of ships out there with the same sob story as Janeway.
How come Gul Evek got sent home yet Ransom was set adrift?
Of course some say the Vetar avoided the Caretakers grasp, but there's little proof one way or the other.