Not a programming issue, because it applies to virtually every NPC in the game it can be concluded that such was intended, no it was not really different in past games, the problem is NPC's are literally TES versions of Fallout NPC's they have nothing to say past the first line if they aren't related to quests.
it also has to do with the distribution of voice actors, 70 VA's and apparently you only hear 10 of them with the remaining 60 having about a paragraph worth of dialog.
Sounds like it, also you hear the same stupid comments all the time. Everybody hates the female blacksmith in Whiterun as you spend so much time crafting together with her and she goes on about his father. Cutting timber at the sawmill in Riverwood and you started hating Lud, the mill owner who goes on and on.
The special actors are probably reserved for quest characters you only meet once.