Try sheathing your weapons.
^This.
If your weapons ARE sheathed... draw them out and put em back away.
Guards will almost always stop (unless you've been on a guard killing rampage

) when you sheathe weapons and attempt arrest.
PROTIP: If you have ANY stolen items when a guard comes to arrest you... say "ID RATHER DIE!" then run, interact with a barrel (remember which one!) and store all your stolen loot. Draw your sword and when the guards catch up, put it away and let them arrest you. After they sort out your bail, walk back to the barrel to get your booty