So i'm making a new character and i've got this idea for a LOTR "gandalf" type build. I'm know people have done it and there's mods but i'm playing on xbox. I wanna go one handed sword and staff (like gandalf). I was thinking alteration for the armor and paralyze, then using the one handed to cut enemies down and do damage. Then eventually i'd also like to fill up restoration (gandalf the white . I'd post a link to the build but it's not letting me for some reason.
I'm just wondering if it's even possible to survive with mostly cloth armor using that build considering i'll be using a sword and staff. I really don't wanna wear heavy or light because it wouldn't fit my idea. I wanna stick to robes at least. Give me your thoughts Skyrim junkies.

You best play on Novice. As fun as that would be, it is not easy.

I made a Khajiit StealthMage, he wears light armor, and THATS hard.

True, although I know from experience with enough perks and enchantments even the most wacky builds can work. My main is a 2 handed sneak assassin and kills everything on master difficulty. I'll probably go into destruction too since it's also relevant to the gandalf lore.

I think this could work but you'll need to rely mostly on your staff unless (or until) you get to high levels of Alteration. At lower levels you can find or make some really good staves to cast firebolts and summon Atronachs so you can stay out of melee range. Once you get Alteration high enough, you can get short term protection as good as the best armor.
The worst part is that you'll have to wield the staff in your left hand and that can be awkward with the quickkeys since in my experience staves seem to revert to right hand which means you have to click several times to get you weapon where you want it.

My idea exactly. I wanna get a good staff of paralysis and max out alteration then use the staff and sword to kick ass. It's going to take a lot of perks though.. sometimes I miss the old oblivion leveling system.

Only one problem with your idea. You have to use spells first before becoming the badass that Gandalf is.

Staves work as well as the strength of the skill tree you have.

If you're a novice in Destruction, you will be casting weak spells from a staff. However, if you're a master in Destruction, you will cast much stronger spells with the same staff.

As far as swords go, mages are not made to take damage or even get close to their enemies in Skyrim. However, as Laxador said, you just need to play on a smaller difficulty, like Novice.

Another thing about swords is you have to use them to become good at them. If you're wearing cloth and using swords, you either have a death wish, or you're playing on an easy difficulty.

Other than that, I've been wanting to try out the Gandalf idea too, but never really put it to practicality. Hope it works out for you.

Another interesting thing is I wish you could quick-change spells in your staff and make them have multiple uses.

Up-Down on the xbox D-pad is favorites, just make Staves left-right? Eh. Oh well.

I die a hell of a lot unarmoured on Expert, but I don't mind playing that way. It makes you try everything to kill that bandit chief or deathlord. At higher levels you get one shotted by some enemies anyway, but at lower levels I got great use from fortify health enchantments and potions, unfortunately neither fitting Gandalf very well.
Unarmoured melee is about the hardest way to play, a world away from illusion, conjuration and sneak archery. My advice is bite the bullet, do a 'similar to Gandalf but not quite' character, and whack some loving into alchemy or enchanting. At least with enchanting you get to name your ring 'Narya', "for thy labours and thy cares will be heavy".

Yeah that's why i'd be using an alteration staff if I tried it, and i'll likely fill up some of the destruction tree too. I don't wanna be up close constantly, but I want that option to use the sword if I end up close. I'm running bleak falls right now on adept and i'm actually doing okay so far with a cloth robe and light boots. I'm just healing myself back up and then switching back to fire and sword. I wonder if armor enchants might be worth looking into.