I don't use Smithing. I haven't even invested a single perk into it. Sometimes it's annoying, because my Smithing is only 33 now and if I want to upgrade my Blade of Woe (which requires no materials/ingots) I can't because I lack the proper perk (unlocked at level 60). But overally seen it's perfectly viable and makes the game more challenging. I use the x15 damage perk on a stealth character, which pretty much allows you to instakill every living being, although this does mean you can NOT be detected at all or you have a MAJOR problem, because in battle it's still hard sometimes (especially annoying mages throwing ice at me).
just do it, if you don't like it you can always quit.
