The dude just keeps doing laps around whiterun.
Does he ever stop somewhere so I can train and steal the gold back?

AFAIK, he just keeps walking. You should be able to sneak along behind him and steal it back though.

I found him sitting in his house once but that seems to be a rare event. With the latest patch, though, he always seems to be walking around.

Remember that even when the Eye shows that you are detected there is quite a good chance you will still be able to pickpocket, just save before you try.