Talos is not a divinity. He's neither Aedra nor Daedra. But he could very well be an aspect/incarnate of a divinity.
I think it's unclear whether any mortal can ever really ascend to the divines. The Tribunal tried, but they only achieved their might by parasiting it off the power of Lokhran, and casting them down was as easy as severing their link to that power.
To them that’s not really the important part, all gods can be human for all they care. The problem is that according to lore, Talos is doing all he can to maintain the status quo in the world and Thalmor want to unmake the world thus they need to get rid of him or atleast drastically weaken him.
^This^
The Thalmor might even actually believe Talos is a divinity, or a very powerfull aspect of a divinity. Why else would they even care about Talos worship if their ultimate goal is to undo the trickery of Lokhran. (Destroying Nirn, re-ascending their people to their "righfull place.) It's quite possible they wish to severely weaken him by putting an end to people worshiping him. In the grand scheme of the Thalmor their claims to aspire religious purity by inquisition seems like a pretext to me, a disguise for their real intentions.
Add to that, that by removing Talos/Ysmir from the Imperial/Nord pantheon they remove the "spititual" motivator for them as a people to dominate. Possibly demoralising them further if you consider Talos/Ysmir to be the deification of the empire itself, it's ideals/goals/etc, and not just the deification of the mortal Tiber Septim. Besides that there's also an obvious strategical/political reason for removing Talos/Ysmir from the pantheon: it will cause division within Imperial/Nord society, factionizing them, possibly triggering civil wars and instability.