Thanks guys, that clears it up. I'm only lvl 20 now, first play through, going full mage. I'm not against wearing some light armor, but I could never tell which way was better. Then throw in trying to figure out if the Alteration perk was worth it and I was really confused. I'm using all the mage skills and Alchemy, so I don't have any points in, nor do I see being able to put points in, Smithing or Light Armor.
Whether a mage should wear armor or not seems to be mostly a roleplaying choice since you make a good build either way and there doesn't seem to be much penalty for wearing armor other than the mage armor perk.
But you should know that the mages robes with the dual enchantments of reduced casting cost and increased magicka regen cannot be duplicated on armor with the Enchant skill because the base increase regen rate is only 10%, so you can buy master level robes that reduce casting cost of a particular school AND increase magicka regen rate by 150%. You cannot put this enchantment on armor because when you disenchant such robes, the base rate for increased magicka regen will only be 10%. Even if you use the 100 Enchant perk to dual enchant a reduced casting cost and increased magicka regen, the base for increased magicka regen is only 20%. So, you will never get anywhere near 150% increased magicka regen on armor (without seriously exploiting the alchemy/enchant loop).
This is another potential advantage of robes over armor. However, if you max out Enchant, you can get free casting in two schools, which may reneder an increased magicka regen rate somewhat moot, so in the end it is mostly a roleplaying choice.