Has anyone tried wearing enchanted robes or clothes instead of armor, and getting really good with their shield skill? It's actually pretty effective.
It requires you to actively defend, as opposed to reap the benefits of passive defense from armors. But it could be pretty effective I think, especially for mages and thieves

just clothes are underpowered.
Wearing armor ALWAYS has more benefits. You can enchant it aswell, faster stamina recovery than unarmored, chance to avoid/reflect attacks and stuff like that

Getting that far into armor requires precious skill points.

you mean perk points?
Which you get faster by just wearing the armor?

Spending perk points in light armor means you can't spend them in something else, since you have a limited amount of skill points to spend (theres a max level)

If you really want to use everything to getting to the max level, you will have to wear light armor and heavy armor anyways.
Therefore, if you dont wear armor, you will always do worse.

I believe if you max out all skills and do the relevent quests, total of 85 perks are available, which is more than enough.

light armor perks also force you to wear ugly helms. Just sayin'
If you wanna wear nice helms, it's thief or dark brotherhood stuff that you can't enchant.

Nah, I like all the light armor helms. Thats entirely subjective.