Well, here's what I did to fix it, and it totally worked. All of the guards now act like normal guards, and nobody gives me the stupid dialogue loop. As well, my bounty for Markarth went away.
Step 1: Become Thane of Markarth. I realize that doing this is also bugged in itself, but if it's working for you or you've found a way to get past it then you need to do it.
Step 2: Go to the Shrine of Talos and engage the quest dialogue loop. Select the "What did you do to Eltrys?" followed by the "I'll go quietly" option. Don't press 'B' to exit out of the conversation at this point, just let it close itself. The guards should start attacking you.
Step 3: Do not attack them back! Just SHEATHE YOUR WEAPON and run away. Go out of the Shrine and try to find a normal guard - he should try to arrest you.
Step 4: Select the "I'm the Thane, let me go at once" option. He'll apologize, and the guards that were previously trying to murder you will lower their weapons and walk away.
You should now be able to talk to all of the guards normally, and you can re-enter the Shrine of Talos and the "quest guards" won't be there!
I must say that this worked for ME, and I did it twice just to be sure. I really hope this solves the problem for others too.
EDIT: RageGT found another solution using a similar method.
See, I had done the quest and it was alright after I escaped the Prison Mines. But long long after that, in some small village, I cleared some Mines from Blood Silver Mercenaries and a Markarth guard came to arrest me. I submitted and it entered the loop. I escaped conversation then and the guard was calm. But when I reached Markarth again, even near it, all guards would come with the arrest dialogue and loop it.
I had to sneak/invisible into the Shrine of Talos, the quest guards were there and attacked me, I left, the other guards engaged in the conversation and I chose "Pay the Bounty" ... all normal Markarth Guards were reseted to neutral faction against me. I am a Thane of Markarth alright but I chose to pay. Then I entered the Shine again and the quest guards were still there and attacked me. I left, they followed. The other guards remained neutral. So I entered the Shrine again, chopped off a pair of heards of those stubborn arses and it is all cleared!
I had to sneak/invisible into the Shrine of Talos, the quest guards were there and attacked me, I left, the other guards engaged in the conversation and I chose "Pay the Bounty" ... all normal Markarth Guards were reseted to neutral faction against me. I am a Thane of Markarth alright but I chose to pay. Then I entered the Shine again and the quest guards were still there and attacked me. I left, they followed. The other guards remained neutral. So I entered the Shrine again, chopped off a pair of heards of those stubborn arses and it is all cleared!
And here's another variation of the solution:
This has worked for me with just one difference...
I left the Shrine of Talos, ran to the entrance to Markath and spoke to guard there.
Selected the "Thane" answer, he sheathed his sword but the Shrine guards will still showing as hostile.
I left Markath, the guards outside were NOT hostile...so I re-entered, talked to a "regular" guard.. all ok..no "Wait, I know you" or "Stop in the name of the Jarl"
BUT
The Shrine guards were still hostile. Out of frustration I decided to kill them..but this didn't turn anyone hostile NOR did it increase my bounty.
I now have a zero bounty in The Reach and no one is trying to arrest me!
Thanks for this OP
I left the Shrine of Talos, ran to the entrance to Markath and spoke to guard there.
Selected the "Thane" answer, he sheathed his sword but the Shrine guards will still showing as hostile.
I left Markath, the guards outside were NOT hostile...so I re-entered, talked to a "regular" guard.. all ok..no "Wait, I know you" or "Stop in the name of the Jarl"
BUT
The Shrine guards were still hostile. Out of frustration I decided to kill them..but this didn't turn anyone hostile NOR did it increase my bounty.
I now have a zero bounty in The Reach and no one is trying to arrest me!
Thanks for this OP