Is it pretty much an all or nothing skill. I mean like do I have to go all out with it or will I be okay using armor and alteration?
That could be an either/or thing.
You could go for armour but you can also rely on alteration alone, which has the advantage that it will level faster.
The higher level armour spells and the perks are a must then though and you might also want to look into the restoration perks that increase magic resistance.
Alteration is easily raised once you get to level 40 and can cast detect life. the more npc's you detect the faster you will level, so use it in a big city at daytime.
The fastest way to level 40 is probably paying a trainer for 5 skill levels per level.