Plus, a united Empire is a bigger threat to the Thalmor than an ulfric controlled Skyrim.
If the Empire weren't the Thalmor' actual "little friends", I'd agree with you. But the level of bootlicking the Empire has with the Thalmor is just pathetic.
Besides, I really doubt the Thalmor would ever allow the Empire to recover his military power. And with the most renowned warrior-ish provinces out of it (Hammerfell/Skyrim), and with Morrowind destroyed, and Elsweyr/Valenwood in Thalmor's hands...
The Empire is gone, face it. The brightest hope, if you ask me, is an alliance between a free Skyrim and Hammerfell, maybe with also the help of High Rock.