I then went through the entire quest, killing Madanach in the process, stealing his key, and simply sneaking through the exit. I got the ring at the end of the quest and everything seemed to be fine.
I went back to the shrine of Talos however, and the guards re-initiated the quest for escaping the mine.
The wiki says this will only happen if I tried to flee the guards the first time, but I didn't do that. I yielded the first time.
I didn't see any other guards on my way into the shrine, so I don't know if they're all still impacted or not, but this is a pretty serious little issue Bethesda has here.

now every time I try to get in to Markarth I am chased or killed by 10+ guards that won't talk to me. When I can get one of the guards I get stuck in that conversation loop.