Don't use Smithing and/or exploits on your next character and I'm sure you will find the game more enjoyable. I havn't used any crafting skills on my characters for a while and the gameplay experience is certainly better for it. I wish I didn't have to ignore features though.
I think you can still get reasonable challenge if you use crafting skills without touching the Restoration loop. The way Fortify Restoration increases the magnitude of enchanted items just smacks of oversight or something that should be outright removed.
That said, you can still make obscenely powerful weapons with a 130% Fortify Smithing potion and maxed Blacksmith skill + perks. 120+ damage on your weapon, plus 90 in Enchant damage can make for some quick kills. (Look here http://www.uesp.net/wiki/Skyrim:Smithing and here http://www.uesp.net/wiki/Skyrim:Weapons for the actual math).
On the defense side, without the Resto loop or Falmer helm stacking bug, you can hit armor cap (80% damage reduction) with three pieces of late-game armor, get 95%+ cost reduction in up to two magic schools, throw on around 2-3 50% elemental and magic resistances, water-breathing, muffle and limited carrying capacity, unarmed or a moderate health increase. Add quest-based perks, standing stones and blessings and you can definitely make yourself an all-around powerhouse, though not beyond the realm of being taken down by multiple enemies, guards or dragons, and that's on Adept.
To achieve that, however, you need to take around 28 perk points in crafting skills (at least 24), which could otherwise go into adding versatility to your character (Speech, Sneak, Lockpick, additional magic schools, better Blocking, etc.).