Along those lines: can you run and bump him like the archer in Solitude or perhaps Wabbajack him?
I am ashamed to say I've never used the Wabbajack...

From my experience anything that staggers an NPC, but doesn't aggro them can work for certain 'stuck' situations. If it's your follower you can even shout at them, since they won't be offended, but Ulfberth is probably not that tolerant - especially if he's standing there engrossed in thoughts of Lydia and then you yell at him.