The episode where this is explained is called Tattoo from season 2.
The aliens who acted as the benefactors to his ancestors on Earth were originally from the Delta Quadrant. So when they meet them. These aliens are about to kill Voyager's away team, when they pause and say "Wow! What a coincidence! You have a tattoo just like our tattoo! So maybe we won't kill you after all." The rest of the episode was about a flash back to when Chakotay was 16 on a bush trek with his father trying to get in touch with their roots and sentiments, but instead they run across Alien hybrid Indians who have never seen the rest of the 24th century, who give Chakotay and his father tattoos.
It's just awful.
I do like the actor who plays his papa though.
He was the dad on the Duncan Regehr Zorro from the 90s.