How do you figure? The Thalmor couldn't take Hammerfell. I think of it as, the Thalmor would have to fight multiple fronts if the Empire were dissolved.
If the Stormcloaks were in power, they would seal off their borders and go defensive. The Redguards are also not invading, they are just keeping the Thalmor out, that leaves the Dunmer (who are a joke), The Argonians (who most likely suffered major casualties defeating the Dark Elves and even 200 years later haven't recovered), The Khajitt seem to be rather divided, like the kender of Dragonlance (could be wrong, who knows), Valenwood is with the Thalmor and the Empire would be almost destroyed after trying to stop the provinces rebelling.