If Skyrim's voice actors are so "Diverse" and "varied", why do I hear 30 nords that sound like Popeye Vogelsang?
Poor management.
Basically....
Take New Vegas for example. Unique, important NPCs got unique, often higher quality voice actors to voice them, whereas nameless, anonymous NPCs got normal voice actors that would repeat.
Skyrim thought that was a pretty good formula, so Skyrim took it too.
The problem? Unique and important NPCs in New Vegas often offer 20-40 minutes of conversation material (in one sitting), unique and important NPCs in Skyrim only offer 2-5 minutes of conversation material. Molag Bal for example has a unique voice actor, but you're really only going to hear him for his brief quest, then you never hear him again. Even NPCs you can repeatedly visit like Ulfric and Delphine don't have a whole lot to say; not much more than Farkas or J'Zargo.
The formula simply doesn't work for Skyrim, but they failed to realize that. So while there ARE unique voice actors, their appearances are VERY brief.