If you have a decent Speech skill and / or a good reputation in any given town, you can actually evade fines or arrest for small crimes. If you have a very small bounty, choosing the appropriate speech option actually gets you out of trouble with the guards. They basically tell you "well, your fines are pretty small, so I guess you can go... but keep your head down," etc. If you've done good things for the town or the locals, they sometimes even bring them up. Or they'll acknowledge that you are - or might be - the dragonborn, and let you off with a warning.

It's a neat touch.

As for melodramatic / memorable guard one-liners, I have yet to actually be spotted by a guard while committing a crime. I can say, however, that the guards will often recognize you if you pass near them with a bounty on your head. They say something to the effect of: "You there - I recognize you. You're a wanted man." The guards "spot radius" may increase if your bounty is larger, I don't know. Just a feeling that I have, but it would be cool.

All in all, the "crime & punishment" system is MUCH improved from Oblivion - it feels much more natural. If a bunch of people witness you committing a crime (i.e. if you openly steal something or kill someone's livestock), it can still bring a silly amount of attention (i.e. an angry mob) down on you, sometimes. But other than that, the reactions of people and guards are much more realistic, and news of a crime you JUST committed is no longer magically broadcast to every living creature in the village, now. If you quickly kill everyone who saw you do something wrong, you can prevent the guards and villagers from knowing what you did, and going aggro.