Did you happen to do any of the following:
Sided with Kematu and captured Saadia in "In My Time of Need"
Killed any Thalmor patrols.
Did the quest where you rescue the Greymane from the Thalmor keep
If not, then it may be the fact that you have been discovered to be Dragonborn. By virtue of being Dragonborn, your very existence is a threat to their plans to unmake reality so they plan to kill you.
I haven't had Thalmor assassins show up yet. Or at least I don't think I have.
* I didn't buy Saadia's story. I mean if she spoke out against the Thalmor in any territory held by Thalmor she could have fled to a resistance cell in Hammerfell. No. Liar liar pants on fire.
* Well, there was that Thalmor patrol I ran into outside of the Thalmor embassy on my way to this cave. (I knew I shouldn't have made that left toin at Albequoikee). Messy business there.
* And then during the investigation of the Grey-mane's brother, I told him to stay home and I'd handle it without bloodshed, but my sword hand twitched and damn if I didn't have to wipe out the entire fort. That was some messy business. Amazing how quickly word travels, too. It got back to Whiterun before I did with the guards congratulating me for giving the Thalmor a black eye.
and there's that one in the College in Winterhold that must die. He is in Ulfric's territory.
There was that Thalmor patrol I ran into... and they did attack, but they were being chased by a cave bear and spriggan and I did a fus ro dah on them. Do you think they were after me?