After the Red Year, most Dunmer fled to Skyrim. Many still went to Solstheim after, but at any rate there should be far, far more of them. there are less Dunmer NPCs in Skyrim than there was in Oblivion (source: http://www.uesp.net/wiki/Lore:Dunmer_Names#Male_Dunmer_Names)

We need more Dunmer!

Dragonborn is coming, and there most likely going to be more Dunmer.