I've played a new Concept; Dunmer Vampire. I chose dunmer to help with the fire resistance (and vamp's weakness), but fire hasn't come into play as much so I wager I could eventually roll a different race to be a vampire.
Unfortunately, I was forced to powerlevel her through Hermaeus Mora's quest reward because I chose to be a hybrid assassin/caster. The caster part still troubles me for Skyrim. Enemy difficulty spikes faster than my magical ability ever does. Mana is rarely the issue, just the power of my spells. I still deal with Flames and Firebolt in my 20's, which eventually does very little and saps my mana for more difficult fights.
Meanwhile, even in the 20 range, I am unable to actually afford all the better spells. And even then, my skill isn't high around (Destruction possibly being in the 40-60's), so not as many perks invested to help cut the cost of the stronger spells, which DOES lead to mana problems. I'm just not sure how to level a caster properly.
Even with the Mage stone, magic skills just don't level up as rapidly as melee or archery. But all of that levels fine, so when I resort to a one-handed sword when magic is failing me - that levels up several times, which contributes leveling while magic is, yet again, left in the dust.

This isn't even a catch 22 situation. No matter what I do different when I try to play a caster - it always, eventually, becomes too damn difficult. I'm eventually not even a glass cannon. I'm an annoying little gnat when faced with dragons, hired thugs, bandit highwaymen and pretty many any other mage enemy.

What am I doing wrong?

I think the problem is that you've leveled too many skills so that you're at a high level but your skills are at medium levels. When I make a mage, I do as much as possible to only level the skills I want, usually destruction, enchanting and conjuration. By level 20 I have destruction and enchanting both at 80ish so I can make 96% cost reduction gear and use the best spells.
With a companion and a summons, you should have no problem with bandits. Also, ditch the mage stone and get either Lord or Atronach for the resists. (definitely atronach if you're a Breton for once a day total immunity)
Making a hybrid mage is something I've never been able to do, warrior with a little conjuration, sure...but evenly balanced? I don't think it's a good idea.

edit: By hybrid, I mean in skills. Mage/assassin is no problem as an RP but stick with mage skills to do it.

i would say use your non combat specific spells when not in combat.

muffle, bound sword, oakflesh, summoning etc can all be used.

that will increase your level in these specific disiplines.

that will increase your character level and let you take more perks.

My hybrid thief/assasin/ illusionist is just fine, since he does his damage with bows and one-handed weapons. Damage with destruction spells appears to be tough in non-specialist mage hybrid?

Yes, the only way it's worked for me is to specialize so that you can max destruction at a relatively low level. After that, you can broaden out if you wish. If you just use magic for crowd control and utility then hybrids are fine, just don't count on destruction for damage.

^^This.

Destruction is the most important one to level early. Dont Forget the impact perk! That stagger has saved my a$$ more times than I can count. Enchanting is vital to any magic build. If you use it to reduce spell costs, you can pretty much ignore the 1/2 cost perks. That means more perks for that hybrid build. I use Illusion as well to get me through the tight spots, but you could just as well use conjuration. I chose Illusion first for the quiet casting perk.... Absolutely vital to any sneak mage variant.

Check out this build....My Mage thief assassin.

At lvl 51
[link]
http://www.ign.com/builds/the-elder-scrolls-5-skyrim?d=0000000000000000001100001111111000000000000000120202000000110000000000000000000000500011100000000000000000000000000000000000000000000000000000000051000100000000011111111100000000000111000011111
[/link]

At lvl 81
[link]
http://www.ign.com/builds/the-elder-scrolls-5-skyrim?d=0000000000000000001100001111111100010110000000120202011001111011000002001113000003511111100110000000000000000000000000000000000000000000000000000051100100000000011111111100000000000111101111111
[/link]

Note not a single combat perk. You don't need them if you use enchanting and alchemy to boost your smithing( no restoration loop required ). I have an eleven bow that does 137 without any fortify archery enchants. The key to the Mage hybrid is enchanting. Enchanting allows you to be more flexible with your perk points. The only way to boost destruction damge output is with alchemy. Alchemy is supposed to be the stealth crafting skill, but it benefits a Mage tremendously. I do admit though that once I am spotted, I am in trouble...... Invisibility is your friend! I put almost all points into magicka final 81 stats are magicka 700 health 200 stamina 200. Health and stamina were fairly early choices, and then nothing but magicka.
This character has been a lot of fun to play.

The thread in my signature has a lot of discussion. On vampire hybrid builds I think you might find it interesting.

Instead of destruction why not concentrate more on illusion for fury/calm etc its very powerfull and don't.forget illusion gets a bonus from the vamps powers.

I would focus on illusion first then as long as u have a magicka potion you'll have no problem laying a frost rune or fire rune trap to help gain distance if needed you cab then always pummel then with fire balls after if u like/need to. Also as your wandering around cast candlelight and muffle.from illusion and flames.from destruction that will help u skill them up faster.

And u have no need to pick races based on bonuses I'm an argonian vampire following the.builds mentioned in the thread In my signature. Also as AC vamp maaje sure u take necromage.

Sorry for spelling and.punctuation. posting from my mobile.

I leveled illusion to the 80's on my thief assassin just by having muffle in my left hand constantly. Sounds cheese, but if I have the mana, and I have a spell that benefits my character type (sneak), why wouldn't I have it up ALL THE TIME? Now, for destruction this may be more difficult, I don't know. I can't wait to try playing a mage. Just have to decide when to quit this thief assassin playthrough (only major questline I've completed is thieves guild and I'm level 55...sigh) and whether or not I want to play a warrior or mage next.

Only focus on a couple skills like destruction, restoration, and conj.
Then focus on something a little less mage like one-handed or marksman.
That should even out your character so you wont become a high level to fast and not have the gear to battle the tougher enemies.

Destruction Help: Get the dual-casting perk, get the impact perk. Use fire bolt or ice spike on every single enemy you come across. If you fight a fire dragon use ice spike on him, fighting ice dragon use fire bolt. Fire and Ice are the best to use because nothing in the game really has a weakness to shock, but a TON of enemies are either weak to fire or frost. Use fire on draugr, trolls, vampires, ice dragons, frost atronachs, nords, etc. Use ice spike on flame atronachs, fire dragons (alduin is fire), and others that I can't think of right now. Always dual-cast it, and use it on every enemy. Your destruction will raise dramatically. With the impact perk you don't have to worry about switching to melee as they get close, just stagger them and keep backing away firing.

From the sounds of things, casters have much more of a 'prep phase' to go through than other playstyles. I don't want to blow 5 hours before I can actually start playing the game normally :/

Just a couple perks and 2 spells and your ready to go. Not that much prep if you ask me.

Correct. Starting out as a mage has been easier than as a warrior for me because the low level enemies are mostly quite weak.
If you are a Breton, for example, you start with flame and have enough conjuration to get a flame atronach as soon as you can afford it. By level 4, you'll have all novice destruction spells (and novice perk) and a flame atronach (also with perk). Add the atronach sign for once a day total magic immunity and you can go just about anywhere up to about level 10. I just did forelhorst with a character like that at level 6.

I look at it as "frontloaded difficulty".

For me, the single biggest factor in my successful mage build was maxing my enchanting and getting the double-effect perk. With that you get enough enchantment slots to reduce the mana cost of spells to 0 in two schools. I picked Conjuration and Destruction. Once that's done you can spam destruction spells with impunity and cast the biggest baddest conjuration spells with a minimal mana pool. That has allowed me to put more points into health, which increases my survival. Once Destruction was high enough, I grabbed the stagger perk on double-casted spells and its pretty much all down hill from there.

Prior to that tipping point, I frequently wondered why I bothered, since my sword-and-board character could deal with baddies quicker. Now its the opposite. I cast ice storm into a group of baddies and they're staggered long enough to get hit by the second wave, which staggers them long enough to get hit by the third. If they ain't dead by then, then at least they're still staggered. Dragons are even easier than deer, since deer are smart enough to run AWAY from me and avoid that first staggering blow.