As I said when he got imprisoned...he got six years for nothing. That'll radicalize anyone. He doesn't need it to be something typical. That HURTS, that is relatable, that is an injustice. That should get anyone on the path to rebellion. It's not even, 'oh I was part of this protest, or I was doing a crime', he got abused by the system personally. It's seeping its tendrils everywhere. I dunno if Niamos had the reputation of being a libertine respite of the galaxy, but judging Andor's character, I doubt he would had went there if he knew there was some reputation of it being a trap for regular people, so it comes as a shock. Hell, he might had been fine with six months, even, but six years???
Something to think about: the way the Galactic Empire imprisoned him arbitrarily and then used him and his fellow prisoners for slave labor has some very notable similarities to the way the United States legal system treats black people and Latinos.