Okay, I originally read the initial post and went "full stop! abandon ship! Here be monsters!". My knee-jerk reaction to broaching such a subject would be that it's a very bad idea. Not because I think the idea is bad, but because of amount of flak I think someone would get if it weren't handled absolutely perfectly.
As such, I would approach the subject carefully and think of all sorts of valid reasons why such a world exists and how it could make sense so that it's not just "about slavery", but slavery is merely a part of it.
The way I would approach your idea is that these criminals, who are now slaves, have to earn something in particular from the guards in question in order to be "freed". Like say, you take these members on Bounties or other things and upon returning the Bounty, they would obtain a "Merit Badge". You could have different types of Merit Badges for different kinds of crimes and even different kinds of Quests/Dungeons. When the slave turned in enough of these to the Guards, they would be forgiven their crimes and marked as free with maybe 50 gold in their hands to start a new life. Some of these now free people could then be hired on permanently if they liked or respected you enough, or even as a means to help you find more work for your other Slaves. I would make sure that it is understood that your main character is doing what he/she is doing for the altruistic reasons of freeing and reforming people. You could probably even throw in plot twists like some freed people end up back as slaves again because they weren't reformed when you let them go, and that maybe doesn't reflect well on you. Or, maybe, despite all the effort, there are just Slaves the Government won't allow to be free because of how valuable they are as commodities. In such cases, you could have the Government take possession of them once again as soon as you completed their last job.
Likewise, I would also make sure to have several of every possible ethnicity, gender, and even animal as slave. Slavery would be equal in the society in that it isn't based upon anything other than committing a criminal act, and thus, there's no dominant archetype of criminal. They come from all backgrounds, walks of life, and even locations.
Even after that, I would still be careful on how to proceed. It would be real easy to botch such a thing and get an in-box flooded with hate.