Actually, this makes perfect sense now and doesn't seem like a glitch. If you attack a guard and help the forsworn escape...you'd be considered a forsworn, right? So the guards would continue to harass you. I don't think this is a glitch, in all honesty.
See:
It is a glitch and does happen to everyone just not on the same level there are two types of people coming out of the mines
type 1, those who did not fight along side the forsworn they are free to come and go in the city, but if they get caught committing even the smallest of crimes the entire cidhna mine loop starts all over again forcing a reload of the previous save.
type 2, those that fought along side the forsworn when leaving the mine, and cant step into the city without getting attacked and starting the loop over again.
Here is a fix that will work for both types of people.
To fix on XBox and PS3, make sure your level is high enough and enter Markarth. Upon entering the city, do not have any guards spot you, and enter the Shrine of Talos. The three guards from the previous quest "The Forsworn Conspiracy," will still be there, and the middle guard will engage in the same conversation as at the end of that quest. The other two guards will attack you, and because you are in conversation, there is no way to fight back, so be prepared. Answer the guard with "You're corrupt, Thonar is paying you," and then again with "Fine, I'll come quietly." The guard will then do nothing, so heal yourself. After a few seconds, the same guard will attempt to arrest you, and give you the option to pay off your bounty
These guards will be there regardless if you killed them during the first encounter or not they respawn.
Seems there is some controversy here - it sounds like the poster below helped the forsworn, and is not having any trouble with the guards.
Finished it without any bugs, escaped together with the Forsworn fromm Cidhna Mine, just didnt participate in the Fight in town afterwards,
Personally I think it is a glitch, considering there are even some people who never do anything with the quest other than see the murder and get approached by Eltrys, and even they get hassled by the guards sometimes. Also, the fact that the entire quest is started automatically as a scripted event when you enter Markarth. I don't think a quest that could prevent you from becoming Thane or completing several other quests within Markarth would be automatically started upon entering the town.