United We Stand, divided we fall. The best chance for the empire to take out the Thalmor is with a United force. Certainly if Skyrim wants to follow Hammerfell and leave the empire, they can but that will make it all the easier for the Thalmor to conquer them all.
There's another saying in statecraft- the more concentrated power is, the easier it is to conquer (James Boswell). The signing of the WGC proves this pretty aptly. If the head of a monolithic power is weak, the whole body suffers. Where power is decentralized and local, resistance is stronger. This is why Rome could conquer Gaul but not Germany. It's why Rome itself faltered during times of civil war over the succession. By interfering in its provinces and taking away their local initiative, the empire is weakening everyone.
Unity under a single head only works when that head is both strong and flexible so as to be responsive. The current empire is neither.
But this one doesn't have to fall now. Let's wait for when we don't risk destroying the world, then we will see.
IMO it's already gone.