Here are a few things you can do to make it much more challenging.

- Play on Master difficulty. ( This is the most funny one because like 60-70% of the ppl who said how easy the game is said they play on default settings, I mean seriously braindead or what? )

- Don't use potions between fights, only allow yourself to use them once the fight is over not during the fight.

- Don't enchant your stuff, ignore enchantments as if they don't exist.

- Don't upgrade your weapons/armor with smithing etc.. use the vanilla armor.

- Don't wear a helmet, in addition to looking cool and badass you don't get any bonuses from full set armor perks and alot less defence.

These are just a few things you can do if you really want to enjoy more realism and a bigger challenge instead of whining how easy the game is.

To be honost, i didn't bother to enchant my weapons armor since Morrowind.
I never used a soulgem , ever...
And as far as upgrading armor/weapons, i havent done that either. Because in the beginning it was hard to find raw material, and what i found i wanted to save for the best armor in the game.