Do all of it. It's only one skill point per tier and you don't have to make all the armor.
Yea thought about it too, but imho it's waste of perks if you don't even use them to make armor that you don't need.

Basically Its a choice - weapons or armor (in your situation)
When i had to choose, i chose Light armor skill perk cause weapons are easier to find, than all armor set
