I've been trying to use the Fast travel system for a while and every time the guy in Whiterun tells me to climb in the back and we'll head off....

I "E" and climb aboard...... then sit there for an hour(in game) waiting for something to happen all the while he keeps taunting me to "hurry up" or "you done yet?"

What... The.... Hell?

It does seem a little bugged. Try talking to him while in the carriage. Have you paid him the money yet?

It doesn't even remove gold from me... just him saying "climb in back and we'll be off."

I'm using the PC version if anyone's wondering.