I agree. I was wearing fully smithed and enchanted glass everywhere. What drove me crazy was how LOUD it was. Yes I was muffled, so my enemies didn't hear it, but I still could. Sneaking around it sounded like a dwemer centurion stomping through a marble courtyard. Or maybe it sounded so loud because everything else was so silent. Also irritated me because I would sneak past someone and they would look at me but not see me. Being in third person drove me crazy, I know I'm sneaky, but I'm wearing shiny neon green and gold armor, how do you not see me?
Switch to nightingale armor, and since I really still wanted 1 hand enchantment I switched out the NG boots for just regular tall black boots and enchanted those w/ 1hand and muffle. Still looks great with the armor, my feet are still loud to my ears but cloth/leather on floors sounds a lot better than the glass did. Also when in the shadows I look fairly invisible so I don't get mad at the NPCs for being blind. Of course my armor rating is basically cut in half.
I like the higher-tier Light armors, and in fact my characters usually wear Dragonscale, however I do agree that sneaking around in anything past about Leather looks really stupid. I mean, here's my Thief, in Dragonscale and toting a Daedric Bow, creeping along the wall not six feet from a guy on a stool, and he doesn't so much as twitch, when what he
should be doing is freaking out and calling for help. Yeah, the character has 100 Sneak and all the perks, but he still sticks out like a sore thumb.
Protection-wise the higher-tier stuff is easier to cap out, which is the main reason I prefer to wear it. I do have a Thief planned who will stop at Leather, though, since it will be easier to blend in that way.