The Dunmer spread out to Skyrim because they had no where else to go. And the Nords subjugated them, forcing them into the slums of Windhelm. The Nords ask "why don't you fight for the stormcloaks?" Because the whole rebellion, and issue with the Thalmor, does not concern the Dunmer at all! Most still cling to the Tribunal beliefs despite its destruction! They could care less about Talos, or Skyrim's independence. And is a poorly motivated rag tag band of refugee's really going to turn the tide of war?
Cyrodiil was right next door.

So was solstheim which the nords GAVE to the dunmer.
But no you have that horribly wrong. Windhelm took in most of the dunmer refugees. Which completely occupied the snow quarter(Meaning the nords in there were
evicted). They were allowed to be self governing, pay no taxes, and worship whoever they want. They've been there 180 years and the nords don't feel like they're contributing anything to them. They've kept themselves isolated unlike the dunmer that fled to Riften. Their original culture was very antagonistic, so it's no surprise that there's friction between them and the nords of Windhelm.
And yes, they could make a rather large difference in a war where every single man counts.(For Example, the current war)
BTW the Thalmor will genocide the dunmer after they finish with the empire. The Thalmor will genocide everyone except a few of their staunchest bosmer supporters because the other mer didn't follow their ideology thousands of years ago and are filthy disgusting daedra worshippers.