When you get caught, you pay 25 gold fine, and you lose any stolen item in the inventory. Then you can just go back to the same person and try pickpocketing that same item again. If you have any stolen item that you care about, you just need to put it away before attempting pickpocketing.
And pickpocketing skill levels very quickly.
I was worried about being barred from pickpocketing the same person again after getting caught, but that was not the case. Just bandits and guards do that, I think.