No one is claiming they *cant* clear a dungeon. It's laughably easy to do in Skyrim with any decent gear.
The issue is that without Enchanting, resist gear, Shield Perk, shouts. It's artificially harder.
What does that mean? It means that you can't really roleplay a Pure Fighter type, without taking a severe difficulty penalty. And that hurts roleplay. You spend your points on Health and STA and Weapons and Armor perks....
But because you don't have Enchanting, Smithing you are always weaker than someone who does. That's just how Skyrim works.,
(most of) The best Magic Resistance in the game comes from Smithing/Enchanting. Unless you specifically seek out some legendaries.
My toon is tricked out with resist gear and a shield. So I'm good. But I learned that the hard way.
Basically I'm not really playing a fighter, even tho like you I don't depend on shouts/magic much. I still have max enchanting and smithing gear on.
I also use health potions whenever I mess up. It's kinda cheese and not very RPG-ish / Balanced IMO
And it makes sense; without magic-enchanted gear, you shouldn't have resistance to magic. What is your proposed solution? I'm not sure I understand what you expect to be different.