Wheras other kill skills only become better with smithing/crafting/enchanting and can stand on their own without them, Destruction pretty much requires crafting to be useful in the first place. Past the Apprentice missile spells, cast cost/damage efficiency ratio takes a nosedive faster than a dragon that got a facefull of Dragonrend. That and the weapon-based kill skills get (if I remember my tally I did once) ten means to increase damage. Destruction? IIRC, four.
EDIT: Let's do a tally of methods to increase damage for the four kill skills...
Weapon-based kill skills (One-handed, Two-handed, Archery)
- Rote skill increase
- 100% boost for all forms in a skill type over 5 perks (Armsman et al) for each skill
- Graduated weapon material tiers
- Power attacks for melee weapons
- Random crits for longblades and bows
- Bleed damage for axes
- Fortify Skill apparel enchants
- Elemental damage weapon enchants
- Weapon improvement via smithing
- Fortify Skill alchemy potions
Any detriments to any of those? Nothing hugely debilitating...
- Enchantments slowly drain energy, but that can be solved by enchanting your own and using grand souls, and the game is throwing soul gems, even filled ones, at you left and right as you progress in the game
- Power attack stamina cost is directly related to weapon weight... however, these weapons can still be used with a depleted stamina bar... Destruction is absolutely useless with a depleted magica bar.
Now, what does Destruction get to increase damage?
- Graduated expertise tier spells
- Alchemy potions
- 50% for all forms of damage spread over 6 perks (Augmented perks)
- Dual casting
Note the complete lack of rote skill increase in that.
What are the detriments for those?
- Spell cost/Damage per cast ratios get markedly worse and worse past the Apprentice missile spells, climixing in the utterly impractical Master-level spells
- Dual-casting exacerbates these exorbitant increases in spell cost/damage ratios (2.8x cost increase for 2.2x damage increase)
- The "50% decrease" expertise spell line does nowhere near enough to mitigate this exorbitance, necessitating enchanting abuse to make them anywhere near cost-effective