How is magic weak in Skyrim lore-wise?
I meant from a social perspective.
I was just being a bit anolytic of the lore as history. After the Oblivion Crisis the Mages Guild was disbanded, in part out of fear it [or mages/magic in general] played a hand in the event. Then the Red Year weakened the stature of those at the College of Winterhold. The Empire was weakened at the top due to the Emperor's assassination, but the disbanding of the Mages Guild left the Empire at a disadvantage when the Thalmor rose to power.
Meanwhile, in the home of the Aldmeri Dominion [Summerset Isle] resides many strong mages. Summerset Isle has long been ruled by mages and the Thalmor gained power in part on the reputation they gained using magic in repelling the Daedra during the Oblivion Crisis. Magic is still systemic in the Aldmeri Dominion and it is their tactical advantage.
I think any defeat of the Thalmor requires a shoring up of magic skills into an organized force by anyone that wants to defeat them. Shouts and strong traditional arms can't defeat strong magic with regular troops, this has already been proven by Jurgen Windcaller's experiences at the Battle of Red Mountain. Shouts are either ineffective at achieving victory in large scale battles with these factors in place, or the gods don't take kindly to it being used in such ways as Jurgen ultimately surmised... or both.
http://elderscrolls.wikia.com/wiki/Summerset_Isles
Galmar Stone-Fist mentions after a Stormcloaks victory during the Civil War that an invasion of the Summerset Isle resides in the distant future for the Nords
I just don't think such a thing can ever be successful without a strong contingent of mages and I don't think you can organize such in Skyrim's [and the Empire's] current social climate concerning magic users. This will have to change for any invasion of Summerset Isle to be successful.
The question are "What event initiates this change?" and "Will we be playing it out in some future DLC?"