1 nomatter what the reason the empire should not let the thalmor to abduct citizens
2skyrim wont take the thalmor on by it self Ulfric talks of allices with thw other provices
Problem is the other provinces may not like Ulfric, the Dunmer don't like him, the Orcs still prefer the legion and the Stormcloaks have shown distrust and disgust by the Orcs for living and owning land in skyrim, High rock still supports the Empire and the Redguards most likely wouldn't accept because they know two provinces wouldn't be able to take down the Thalmor as for Argonians they are still isolationists. Ulfric won't be able to build as much support as he wants as the empire could by having skyrim and high rock within their ranks which would mean that the Redguards would see that the only choice for victory would be by allying themselves with the empire in order to fight the Thalmor and the Dominion.
People jabber about a lot of things. Think it through logically. The Thalmor need to eradicate Talos worship, or at least weaken it so significantly that Talos himself is taken out of the picture. Skyrim is the stronghold of Talos worship. So you're saying that they would have left Skyrim alone, even though they were persecuting the Talos cult elsewhere? That makes no sense whatsoever.
Maybe they couldn't prove that Talos worship was going on to the empire and weren't allowed to actually do anything without proof that a large cult or organization, I.E. Stormcloaks, that actually worshiped Talos as a god and not a man, also the Talos ban was an idea because they knew that Ulfric would be a good pawn due to him being a fanatic Talos worshiper and being the son of a very important Jarl in skyrim. This will also explain why they didn't try to push a ban on Arkay and Dibella, both having been human before being ascended to godhood.