Edit: A full set of Daedric Armor + Shield + 100 Heavy Armor Skill + Ebonyflesh might get your armor capped. I'll have to look into that. I still wouldn't recommend it, as being a one-handed Mage will prevent you from using some forms of magic effectively (namely dual casting Destruction and Illusion), but it's probably viable.
Actually I've found the key to be, to keep a shield out when not in combat just in case your character gets ambushed, and to begin combat with melee enemies either opening up with a two handed cast, or charging right into their faces, shield block stunning them, then finishing them off with multiple two handed casts. The key to the idea is always having a shield out when not casting, opening up with a shield bash depending on the situation or if ambushed/in a tight corner, then letting out the chain casts. Another part of this build idea, is specing for 5/5 in one handed, and stamina cost reduction for one handed abilities along with blocking, so the character can switch between a (decent but not great) dual wielder, or a decent sword-and-board type when out of magicka or depending on the situation. One handed spells in general seem just too weak to rely upon all the time, it's really just about switching between a shield, dual weilding, and two handed destruction attacks depending on the situation. Perhaps using a bow later on as well.
The idea is to have a caster that doesn't share the debilitating weakness most casters have of getting ambushed by melee, is hard to kill with physical damage in spite of not having particularly high hp, and who does decent but not great melee damage when out of magicka. The weakness of the build being fighting other casters, only being good with destruction magic, having relatively poor melee abilities, low mobility, and no ability to sneak. To my pleasant surprise though, I recently discovered that the player can get more maximum perk points than I thought, so getting rid of smithing doesn't seem so necessary anymore.