In the jarls house in riften at the back of the throne if you got right there is a room which has a display case and gems in and around. That respawns all the time.
I don't think they "respawn". I think the owner of the stolen items replaces then once they notice they were stolen. I would imagine the court wizard in Whiterun replaces the stolen item with a new one so that he can try to sell it.