I found that at low levels, there weren't enough spells to protect me against sword and board foes. So I have a suit of armor at the ready.
It takes movement, distraction, putting objects between yourself and the attackers, having a companion early on. There are ways without resorting to armor. I never did and survived just fine. The impact perk is a huge help once you get that but even before that it can be done.
To me a pure mage wears only cloth. What I found worked best for me were the spell specific robes. Since I was primarily destruction I went for gear with reduced magicka use for destruction spells. My boots had carry weight on them. My ring, necklace, and head piece were all geared towards destruction magic. I never enchanted my own gear preferring to use what I found or purchased from vendors. Could I be killed easily? Yes I could. To me that's a mage. High damage output, easy to kill. I had fun with mine. It was definitely my most challenging character and the most fun throughout the play through. The balance was there. I never felt overpowered, there was always danger, big fights felt big, but I could still totally dominate one on one or against lower level enemies. It's probably the most well balanced class in the game to be honest if you play it as a pure mage and don't start tossing in armor and melee weapons. Both of my melee characters were overpowered by the late 30's even though I wasn't trying to make them that way. The game just doesn't scale as well for melee classes. They have too many damage enhancing perks and too many defense options.