It's a bit more than that. Harry was originally going to be written out of the show in the fourth season to make room for Seven of Nine. That is until People named Garrett Wang one of the most beautiful actors on television and UPN decreed that he was staying on the show because of it. So for season 4 onwards, they had a character the writers didn't like writing for, played by an actor who pissed off the producers that they were forced by the network to keep. Kind of rules out him getting any interesting storylines.
He wasn't really released. He was on a work release for the mission to track down Chakotay's ship, a similar situation to Michael Burnham. Had Voyager not been tossed into the Delta Quadrant, then after the mission was finished, he'd have been sent back to prison until his next parole hearing, which Janeway promised to put a good word in for.
They didn't "let" him wear a uniform, he had to wear it. Also note that throughout Caretaker Tom has no rank insignia at all. The reason why the Maquis don't wear Starfleet uniforms in Caretaker is because they aren't part of the Voyager crew yet, they still have their own ship until the battle in the episode's climax. After its destroyed, and the decision is made to integrate the Maquis onto Voyager's crew, they wear uniforms immediately.