Well, in Morrowind it seemed like many of the Dunmer lived in caves in mountains, which is correct by D&D lore as well. I just assumed this race should have some form of infravision.
Vos? Tel Vos? Sadrith Mora? Vivec? None of those were caves or underground and they were primarily Dunmer settlements.
Only Ald'ruun had them living underground and that was simply due to the convinenence of the shell buildings they used. And they were only a few feet below the surface.
Orcs in TES are not as cut and dry, though it does seem like the majority of the hostile one's you encounter all reside inside of caves and mountains which makes sense. These creatures should see in the dark also.
Those were Bandits. hence, why they were hostile.........and also why they hid out in caves,
edIt: And you mention D&D Lore. No relevance at all. D&D Lore and Elder Scrolls Lore are two very different things.