I ran into a problem with smithing on one of my other characters. I leveled smithing so high that all my other skills paled in comparison and my character'ss level was twice what is should have been; this gimped my character with the level scaling system.
That's why you don't power level smithing. I never level it higher than my primary skills.
My very first character was a barbarian build like yours. Dragons are hard, but not impossible, I was taking down blood and frost dragons on expert, so the build is possible. You'll have to use the terrain to your advantage, mainly to avoid their breath attacks. But once it hits the ground, I would run in a use a combination of normal attacks and staggering power attacks usually taking big chunks off their health. Sometimes I would also have to circle the dragon while getting in some cheap shots.