Quick rundown of my problem:
-droped off all of my stuff in Breezehome even my clothes before i went to jail
-killed Madanach
-read his note explaing the exit tunnel thru the prison cell
-tried to double back, way back, first, but the gate would not open to fight Silver Guards
-followed tunnel out as directed in Madanach Notes
-killed only one of the two Dwarven Sphere's and left one alone
-exited tp Markarth; was immediately attacked and the dude wouldn't die but guards did not attack me
-fast traveled back home to pick up my armor and weapons I had left there

This is my problem: Now my quest journal is showing I never exited the mine.

What have I done THIS time?

Can someone help me please?

It's a glitchy quest. Why did you store all of your stuff? It's all saved for you.

-droped off all of my stuff in Breezehome even my clothes before i went to jail You will get it back afterwards
-tried to double back, way back, first, but the gate would not open to fight Silver Guards You cannot open that gate. The chains that opens those gate is in the room next door.
-killed only one of the two Dwarven Sphere's and left one alone You can sneak pass them and they will not respond. That all depends on your sneak level.

You must of done something wrong. I done this couple of hours ago.
I got into jail.
Done some mining.
With some talking I got that little knife.
I fought the bodyguard barehanded and won.
I done a sneak attack with the king of the rags.
Loot his body, got the note, then I sneaked out through the tunnels.
Fought off two spiders, sneaked pass the sphere's and I got out.
I had no problems with the fellow, got my stuff back and I got pardon.

IIRC you also receive any item that was confiscated from you when you paid bounty or serve jail before you started this quest.

I knew I was going to jail--- aint that what ya do in real life? Leave all your stuff at home before ya turn yourself in?

Yes. That includes what ever stuff I picked from that quest that dose not have a stolen tag. The note that I read taken from a chest which tag stolen I dropped it.

Sorry for the late reply.

Ok guys thanks for helping me. But im still in this rut, what do I do? The next step in the quest says " Exit the mine", but im out of the mine already. Should I just forget about it?