For brief periods, one ruler has managed to unite all of Skyrim, but the Nord character is one essentially of conflict, and the confederacies never last. The Cyrodilic Empire and later the Septim Empire was able to take advantage of this tendency and recruit the warlike Nords to their side before they became a force of the opposition.
The following is a quote from the Pocktet Guide of the Empire. Essentially it means that without a strong unifiying force, like Ulfric or the Empire, Skyrim will fall apart in individual Holds.
Who is a more stable unifyning force, Ulfric or the Empire?