Would like to know what the Falmer followers in rags were all about at least... A place this mysterious needs a good mystery questline within it. Something Dwemerian obviously

Agreed. I was really surprised the first time I stuck my nose into the Silent City Catacombs and found actual
people inside. Then it turned straight to nightmare fuel when they turned on me as soon as I attacked one of their Falmer "masters."
Besides that, I think the thing that disappointed me most about Blackreach (aside from a glitch that kept causing it to suddenly be daytime on occasion; talk about ruining the atmosphere) was...well, I read http://www.uesp.net/wiki/Lore:The_Falmer:_A_Study just before delving down into it. Which contains this tidbit:
Generations after they first sought solace among the dwarves, and experienced bitter betrayal, the Falmer rose up against their oppressors. They overthrew the dwarves, and fled even further down, into Blackreach's deepest, most hidden reaches.
...
Finally free from the threat of their Dwemer overlords, the Falmer were able to spread freely throughout Blackreach.
I was expecting hordes of Falmer swarming up from the impenetrable deep--that I'd basically be walking into a proper hive. Instead, I got...a giant, a dragon, some spiders, and a good handful of Falmer. No impenetrable deep. No swarming hive. MEH.

I wanted a proper challenge, dang it!