Okay, forgive me if I'm being stupid but I still don't see what you mean by not putting points into bound weapons because you don't use armor...
I don't think there is a correlation between bound weapons and armor. You could be naked in game and would still get benefits of perks for bound weapons. Do you mean you would be using points for perks in the 1-handed or 2-handed weapon trees instead of putting those same points in armor (because you're not using armor)?
I think what the OP means is that he thinks without armor, he won't be able to survive melee attacks (because the enemy will be hitting him back, and he's afraid he'll go down so fast he'll never have time to attack enough to kill the enemy).
To the OP: You said you are perking "pure mage" right? Well, the alteration tree gives you access to magic 'armor' spells - The oakflesh/stoneflesh/etc spells. With perks, I think these spells become very powerful, and effectively replace armor (I mean, it might not quite get you to the resistance cap, I'm not sure, but I think it gets high enough to be very useful).
You'll also have access to restoration heals, alchemy heals and buffs. Let's not forget illusion spells like paralyze (ok, I've heard that illusion spells don't scale well beyond level 45ish, because the enemies tend to be immune because of their level, so at that point paralyze might not be so useful, but until then, definitely). I don't think you have to worry too much about dieing if you use all the tools available to you.