Is there a way to stop magic users from casting? Interrupt them in any way? svck their magicka away?

Electricity, and rushing them to stagger. If your using a shield, the elemental protection perk will work as well.

Tell him a bad joke? No, seriously, anything that throws him off his action. I like shouting.

Stab them, that's what I do...

use silence poison... oh wait.
Paralysis poison will let you get close without getting fried.

Always strive to stagger them. Shock spells also drain magicka, so if you are a caster, also use that. If a rogue, go for magicka damage poisons, or even better, ones that paralyze. As a warrior, you will need to charge them as fast as you spot them or shout them down. Don't let them get an Ice Spike in you or you are done.

Charge them? Charge them how? You mean just "sprint" towards them?

Yup. Sprinting power attack, shield bash in their face, whichever you prefer. If you are a melee fighter, not much else you can do except getting in melee before they slow you down to a crawl with ice spells. Casters are kind of the paper to melee's rock.

Yes, always try to either get close to casters or stay out of their line of sight.

The first word of Unrelenting force works well.

You can do a "sprinting shield knockdown" with the right perk.

FUS!

Pretty much, the best defense against casters is to boost your magic resistance as much as possible, like using the Lords or Atronach stone and doing The Book of Love quest to get a 40% resistance(with some physical resistance) or a 50% spell absorb effect and a 15% resistance. There is a shield perk that helps too of course as it cuts 50% elemental damage, if you have restoration you can use a ward but they're expensive, alternatively do the The Only Cure quest to get a shield that is very good against mages.

Or do The Only Cure for Peryite and you can have both at the same time

*cough* literally just edited them in before you said that .*cough*

Your own magic, firebolt with 2 perks in aug fire, dual cast, and impact. dual casted firebolt with 50% aug will stagger almost anything including dragons and deathlords and is the stunlock option for mages. cast paralyze before they get a ward shield up, or while the ward shield is down. Summon an antronach or dremora to distract them and firebolt/paralyze from behind.

Fireball something next to them or slightly behind them as ward spells only protect against forward attacks, dual casted with impact, if the damage is high enough they might stagger without the perk.

Use a bow with power shot perk, 50% to stagger and goes through ward spells.

Sprint and shield bash or shield tackle, pommel bash with two handed sword.

werewolf sprint and double power attack - usually just outright kills them.

a power attack from any weapon has a chance to stagger as well, but shield bashing or pommel bashing is more effective.