Well, I've been a bit naughty around Solitude lately, and built up around 2,500 bounty by killing some high officials and guards. Now, when my bounty was low, I could just pay the guards off for a lower price (seeing as I'm a Thieves Guild member), but now all I can do is pay off my bounty, go to jail or fight until death. Now I'd be willing to just pay of my bounty, but the problem is that I then also lose all my stolen goods, and half of my equipment is stolen, so I don't quite want that. Would it be possible to store all my stolen goods in my house in Whiterun and then pay my bounty in Solitude so I get to keep all my stolen goods? Or is there a way to avoid this bounty altogether?

Yes, you can do that. They only take the stolens from your inventory.

As for avoiding it: the only way is too stay out of Haafingar.

Surely placing everything stolen in your home and trekking back will work.

Although Ive never tried it and Skyrim has a way of surprising the hell out of me with stupid stuff.

Put it in your house and get landed in prison or pay the bounty. Save beforehand, just in case.

Yah, they only take stolen goods on your person

Anything stored away is safe....

In fact... you can hit 'B' (cancel conversation... this will cause them to chase and attack you) run and drop equipment on the ground while they give chase.

Then pull out a weapon and face them... then sheathe it. They will take this as your surrendering and... usually... offer to arrest you again.

Pay your fine, waltz back to where you got picked up and grab your dropped gear. Things don't disappear from the ground all that quick... Hell, I had a basket I dropped outside my house sit there for like a week game time... so the quick trip to jail to pay your fine is ... fine

I wouldn't recommend trying the dropped goods trick if you're going to serve time though... it's almost sure to be gone I'd imagine.

Either way... saving first is ALWAYS prudent... no matter what it is. About to climb the staircase to Jovaskr? Save... you just never know lmao (ok, slight exaggeration)

But yah... I did that after getting caught doing something once... I took them on a wild chase and dropped my stolen goods where I could pick em back up before letting them take me away... really felt like I was on the run lmao.

Road away on a stolen horse once too... another really fun moment.... Still gotta pay that bounty lmao

Well, you could see the error in your ways and stop stealing things... and, taking them into that town.