I have a personal question following this topic:
Imagine I would invest every level up into Magicka. This way, I would be very fragile. That is what I want.
Next step, Destruction. Sure, they can cause stagger to most enemies but not all.
What about end game dragons? If I have no rock to hide behind. Wouldn't a Ward spell be very useful?
I do this. On master difficulty too
In order of usefulness when doing pure magic:
1) Illusion. You can dodge an entire fight with some well-placed Frenzy spells. Also I take J'Zargo with me and giving him courage makes him serve as a bodyguard longer.
2) Conjuration. Gotta have a body guard.
3) Destruction. Gotta deal damage and Impact makes it defensive as well
4) Restoration - Alteration. Alteration really has practically zero use spellwise until you get paralyze, whereas Restoration I only use to seldomly heal myself or J'Zargo.
As pure magic you don't want to get hit. AT ALL. The first three make sure that happens, the latter two are defensive. The problem with defense on a pure mage is that magicka is limited, so you really can't waste time defending when the resource you consume to defend (and attack) is finite.