That is much the same thing I am currently going through. Initially, I had planned to forgo Heavy Armor. But in hopes of getting my Conjuration up, before then others, I have postponed that skill. I have also foolishly spent several perks into the Illusion skill, as I don't use it as much as I thought I would. From my experience, it seems that one must invest in that entire tree for it to be truly effective. I guess it will all work out by the later levels.
Illusion is very much 'all or nothing', as otherwise it's a complete waste at higher levels, which means it's going to soak up a lot of perk picks. I used it quite a bit on Vasheera, since the idea was to keep them from hitting me in the first place, but at higher levels I was usually better served by sending in a Storm Atronach or Dremora Lord to soak up hits and then firing from range. If I hadn't been using high-end summons as bait it would have seen a lot more use at higher levels, although by then I also had Paralyze, which tended to be more efficacious since very little resisted it and it didn't need an additional perk to work on Undead and Automatons.
If doing a 'no defenses' run, like I was, Conjuration and Illusion are the most important skills early on since they allow you to distract and/or shut down the opposition. At higher levels Alteration comes into its own, with Paralyze and (if allowed) Dragonhide rendering many fights rather easy by themselves. I added Restoration, since I wasn't using any defensive spells at all, but it's not required with careful play.
I deliberately avoided triggering dragons, since I was skipping the MQ, but I occasionally wished I hadn't; I managed to find the first word of both Ice Form and Slow Time, either of which would have been a valuable asset, but with no souls I couldn't unlock them.
------------------------------------------------------
Started the new character tonight, after adjusting the plan a bit.
Name: Thavarei
Race: Argonian (female)
Class: Mystic Archer
She's a rebuild of one of my retired characters (different name, though), who is not really compatible with my mods and also has a rather sloppy build due to not knowing how many of the perks actually worked.
I'm trying something different this time, and using Bound weapons instead of crafted ones; both to save carrying capacity and to keep damage output to a more reasonable level. I'm also using Illusion, despite it needing a heavy perk investment; since my armor rating is going to svck for an extended period (I made it a lot harder to hit the cap, so that the armor trees' perks are actually useful, and am using Light armor), the ability to make them target either each other or nobody at all will be invaluable.
I managed to get my claws on a Bound Sword tome early on, and discovered that 1) you can dual-wield them and 2) power attacks with the pair put out pretty hefty damage for early on. The downside, of course, is that they cannot be enchanted, so I'll probably end up wielding just one and keeping either a heal or status spell in the other hand, since I won't be blocking or using a shield.