On my first 4 characters, I sided with the so-called "King of Rags" only because I didn't realize there was an option not to.
That's why I love this game. On my fifth character I am still discovering new things. I have built up such a hatred for the Forsworn, I finally decided to show that dirty old bastard what I thought about him. Then I notice he had a note detailing the escape plan, and a key to the gate that he and his goons were going to use to escape.
So as Im breaking out of Cidnah Mine (in record time btw), Im thinking "oh nooo, how am I suppose to get my stuff back?" Because in all my previous playthroughs, one of the forsworn goons takes care of that issue. This time, that Silver-Blood dude greets me as I walk out of the ruins, thanks me for everything and then pardons me and gives me my stuff back!
So yea... never siding with the forsworn again.