I have both the bribe perk and have done the thieves guild quests but I cannot bribe guards when being arrested, the option just isn't there. I have plenty of gold, around 40000 yet cannot bribe my way out of a 5 bounty crime.

I don't have the bribe perk, but I get the option to bribe the guards all the time. Go figure.

I think it has something to do with the fact that my bounty hasn't entirely cleared, I am stuck in the "wait, I know you" loop. Is there some console command to fix this?

Well, there's paycrimegold that pays off your bounty. Not sure if it's what you're looking for, though.