So I sort of completed the forsworn conspiry quest, I think thats the name, where you report to the guy in the Talos Temple in Markarth about the people blah blah blah. BUT the guards were all like "Stop snooping around" and I was like whatever. But next time I went into the Talos Temple I was arrested for snooping around and taken to Chinda Mine, and started "No one escapes Chinda Mine"

I completed that quest and then got sick. I went to the Temple to get healed and the guards were still there, and arrested me like I was still in the quest, you know? And I went back to the Mine, but of course nobody was there.

Is this a glitch with the guards still being there after the whole quest was completed?

Yes, this is to be fixed in the next patch.