I must admit I'm also not fond of shorts! Really not my thing at all.
It's a comfort thing for me really. I wear tee shirts, but I'll wear a jacket over it if in public. It's not a conserved attitude, it's just, as I said, a comfort thing. The more of my body covered, the more comfortable I feel, even in the summer my attire in public is a tee shirt or polo, hoodie, jeans, and shoes.