I can't remember much of the final battle in Ulfric's palace so it must have been pretty mundane. Heck, his bodyguard was more of a match than him. He didn't use the Voice either. All I really remember was him asking that the Dragonborn kill him instead of the headsman. Although it saddened me to do so I gave him a hero's death rather than let him be dragged off to the headsman's block.
Did his body clear after the battle? Quite a while after I defeated Ulfric I went back to Windhelm to talk to the new jarl and there was dead Ulfric, still there, more-or-less laying on the new jarl's knees as he sat upon the throne. Really strange. I've gone back a couple of times since and Ulfric is still there, as is the body of his second, the Commander of the Stormcloaks, laying on the floor to the side of the throne. NPCs walk up as if they're just noticing the body of his second, gasp and exclaim, "Oh, my! What happened here?" every time I go into the throne room.
It's all very disconcerting... and really kills the immersion of the game for me in that area.