Fahrenheit or Celcius?
Like Arawos I'm Norwegian.
Now, in Bergen there are little snow but it's cold.
I would never think of swimming in the fjord at mid-winter with -13 degrees Celcius.
But the hardier of us do so, every day all year around.
Like someone suggested, a radiation like system for cold would work.
But Nords should too have this, with reduction as I think I handle freezing temperatures better than folks from southern lands.
You felt cold in -4? Celsius?
Dude/dudess we walk around in -20 and don't feel cold over here

But yeah khajiits should be affected, no other race, plus they would get used to it after time.
-4 Celsius, that`s what my mobile told me. It was a little windy too and I was walking by a lake, so could`ve been cooler than that. Still pretty cold to me. I actually like it, but it did occur to me that it will still affect you in the end.
I`m sure some people are used to colder than that.
IN terms of races affected, I guess it could be all, but surely Argonians would be most severely affected (cold blooded), then Redguards and the rest of the Human races...? I wonder why Khajitts should be so affected as they have warm fur plus wear clothing, perhaps its a psychological thing, cats do like to be piping warm I`ve noticed.
To be honest I`m surprised there`s even one Argonian in skyrim that isn`t dressed like an Eskimo!