I don't remember the name of the quest, but it starts with a committed murder in the streets of Markarth. It should be an act of a forsworn, allegedly.

So I start the quest by finding evidence, running around the city collecting information from certain people. After that, I am to go back the Shrine of Talos to hand in the information, but when I do, the quest giver hans been killed by some Guards. 2 guards, exactly.
They tell me about their little arrangement with some villain kept hidden inside the Cidha Mine of the city, and that I've interrupted this settlement, which is why they suddenly draw their blades against me.
So I kill them and go back out of the shrine, where I'm faced by other guards wanting to take me to prison. HERES the question:

You have 2 choices:
1-reject and fight (I tried that)
2-accept and go to jail (I haven't tried that, as they won't take me. I've offered them to take me as prisoner, but it just bugs)

So... to complete the quest and kill the guy behind all of the killings, you have to be taken to the Mine, right?

Or... I'm confused, I'd like some insight, please.

Faithfully

Load a save before you killed those two guards and agree to go with them.

You need to be arrested at that point in the quest.

I do not have a save before that, unfortunately.

However, if I might return to the city (as I've fast travelled out of the bug fest) they might take me as prisoner?

Hopefully.

If they're still hostile, take out your weapon and sheath it again. Maybe they'll stop attacking.

I'll try that, thanks

Do you have an Autosave where you entered the shrine?

I know a lot of people have had bugs from fighting the guards. I always surrender peacefully and have had no problems with that quest.

Once you are in Cidhna mine you can decide whether to help the Forsworn (you'll be asked to kill someone) or kill their leader. Since you have no gear, you can use magic (helps to have a bound weapon spell if you're a warrior) or use a shiv or pickaxe.

I would recommend reading a walkthrough if you get this one working. You'll see spoilers for sure, but there are things in the quest that may alter the world even if done without bugs. I'll be more than happy to outline them for you with minimal spoilers, but many people have done what they thought was the right thing and have been disappointed with the outcome.

So you'll be totally disarmed when taken as prisoner? (I've never been in jail before)
Will I get my gear back ?

Just your fists and a set of clothes.

Regardless of which side you take, you'll retrieve all your gear when you escape.

Okay

So I gather that, either you fight or help the forsworn

Yes. If you help the Forsworn try to escape at night because (very minor spoiler).

If you fight in Cidhna Mine (very minor spoiler).

The last time I did this quest I was in Markarth on a side quest for Vex of the TG. When I got back to Riften, the first thing she says, "You got arrested!" Funnily enough, I still had the item she wanted.