Oblivion has hugely been effected by Lord of the Rings' succes. Suddenly Cyrodiil needed to become a copy of Gondor and it's citizens became mainly white, whereas the climate of the region (according to lore) was much more tropical and thus the Imperials should have had a tan.
Skyrim brings back the Imperials as they were previously intended. Now there is nothing wrong with letting things evolve, however the changes seen in Oblivion (which took place 4 years after Morrowind) were just unbelievable. Now they've changed things back, but this can be accepted considering 200 years have past and things should look different.