1. Pay a fine equal to your Bounty, if you have the gold. Any stolen items are confiscated, and you are moved to the jail exit.

2. Agree to go to Jail, where you can serve your time, or attempt to escape. See below for more details on your options once in jail.

3. Resist arrest, if you wish to battle your way out of the location. This is also what happens if you try to back out of dialogue with any guard.

4. Bribe the guard. This option is only available if you aren’t currently trespassing, and one of the following conditions holds:

You have committed only minor Crimes, and have the Speech Skill’s Bribery Perk.

You are a member of the Thieves Guild

You haven’t bribed or persuaded another guard in the past 24 hours.

If you pay the bribe, it stops the current pursuit, and you’re able to keep any stolen items.

However, your Bounty is still in effect, and if you speak to the same (or any other) guard again, you’ll turn yourself in, and the guard will attempt to arrest you.

5. Persuade the Guard. If you have a Speech Skill of 75 or higher, you’re not currently trespassing, you’ve only committed minor crimes, and you haven’t bribed or persuaded another guard in the past 24 hours, you can convince the guard to overlook your crimes.

The option only appears once all of these conditions have been met.

Persuasion works just like a Bribe: It stops the current pursuit, and you’re able to keep any stolen items.

However, your Bounty is still in effect, and if you speak to the same (or any other) guard again, you’ll turn yourself in, and the guard will attempt to arrest you.

However, you don’t have to pay for a persuasion, making it a cheaper option if you can meet the skill requirement.

6. Pay off the Guard, if you’re a member of the Thieves Guild and you’ve completed the City Influence Quest for the Hold that you’ve committed the crime in.
There are four City Influence Quests, for Solitude (Haafingar), Markarth (The Reach), Whiterun (Whiterun Hold), and Windhelm (Eastmarch).
The Thieves Guild is based in Riften (The Rift), so you only need to be inducted into the guild (during Thieves Guild Quest: Loud and Clear), in order to pay off the guards there.
The other four Holds do not have City Influence Quests, so this option is not available in Morthal (Hjaalmarch), Dawnstar (The Pale), Winterhold (Winterhold Hold), or Falkreath (Falkreath Hold).
The price to pay off the guard is always half your current Bounty. This removes the entire bounty and allows you to keep any stolen items in your posession, so it’s always the best option if you can afford it. (basically you have to run a bunch of side quests, 5 in each city to unlock the option to pay off a bounty completely)
7. Invoke Thane’s Privilege. This is only available if you’re the Thane of a particular Hold.

You can be a Thane of any or all of the Holds provided you’ve completed the appropriate Thane Task.

If your Bounty is less than 2,000, you can remind the guards of your social standing to convince them to overlook your deeds. This only works once per Hold, so save it for some pretty devious activities!

8. Civil War: As the Civil War rages on, Holds will fall to either the Imperial or Stormcloak factions.