Later, in 4E 171, the Thalmor invadedHammerfell and Cyrodiil after Titus Mede II rejected their Ultimatum. Aldmeri forces would push deep into both provinces, taking many cities, including the Imperial City. The Thalmor then faced their greatest defeat (later known as the Battle of the Red Ring), in which they lost all of their forces in Cyrodiil. Despite this, due mainly to the exhaustion of the Imperial Legions, the Great War ended with the White-Gold Concordat, which resulted in Talos worship being outlawed in the Empire and a significant portion of southern Hammerfell becoming part of the Dominion. However, Hammerfell rejected the treaty and left the Empire, continuing to fight the Thalmor until 4E 180, when the Second Treaty of Stros M'kai was signed by both sides, resulting in the Aldmeri forces pulling out of Hammerfell and a territorial status quo.
Hammerfell was only able to push back the Thalmor because the Dominion was already weakened after the Battle of the Red Ring.
If I remember right after the White-Gold Concordate many people still secretly worshipped Talos and the Empire was fine with that. Until Ulfric made a mess.
Besides the Empire is slowly growing in strength to strike back again. Just listen what General Tullius says about their future plans. Sometimes to reach a greater goal You need to
stand aside and wait for the right moment.