I've gone sword and board with heavy armor, and didn't have much of a problem. I play on expert and do not want to go down to adept.

I am getting absolutely torn apart by death hounds, bandit thugs with their friends, etc. I have perked only light armor, block, one hand, archery, and smithing. I am wearing a leather cuirass, and Dawnguard everything else. Level 20. What am I doing wrong?

Level 20 and only leather? You might need e few perks in Restoration.

I'm playing an Imperial, and I refuse to wear Elven armor. What should I be wearing?

And with the build I planned, there is no room for restoration perks before level 50. I hate perks.

Imperial steel armor and stealth the dogs with the crossbow.

I run a similar character, Doing what you're doing without any magic-based perks does sound difficult. If you don't mind deviating from your original character concept, a few perks in restoration and alteration might make things a good bit easier.

Scaled armor should be attainable at your level. If not restoration then alchemy for potions. Gotta heal yourself somehow.

Glass armor's nice. My Dunmer uses it, and she's a tank. A green, gold,and gray tank.

Glass is the best you can get, but in case you dont want it try get your armor as high as possible with Smithing, go get the DB armor...even with Perks Light armor is not meant for hand to hand...at least not fair hand to hand, if you are going to go into melee you need to favor speed of the kill over everything else so go Double weild, get some perks to speed up your power attacks and make less costly and use potions and poisons to even things out

Either invest in smithing and/or enchanting or put points into the Light Armor tree. Unfortunately you need to get hit a lot to boost Light Armor. There's always mudcrab training.

You need to wear all Dawnguard gear to get the benefit against vampires (also check that you're not one) Bandit Thugs are lvl 9 enemies so shouldn't be a problem, but if you're seeing Plunderers (19) and Marauders (25) then you should stop crafting for a while and raise your combat skills. That will scale back the enemies you face in future to more managable levels.