Yes, I've had this happen once to me. Turns out they were sent by one of those obnoxious nobles travelling the roads, because I stole everything from their pockets after they insulted me for the 99999999th time.

The Thalmor have attacked me a few times, until they finally got a note with them saying how much of a threat I am to them and yadda yadda. I was very honoured to read it, I'm still carrying the note as a keepsake. I'll gladly take them on from now on. ^^ Before the note, they seemed to be a random encounter, as they were replaced by something else every time I reloaded (when I was utterly destroyed by them, yesh :/).
Another, somewhat more annoying attack is that of the assassin. I meet him/her quite a lot, who was sent by Astrid every time (I already know who she is). Well, on the other hand, it's a good sparring partner for my dual-wielding sword character. And it's not the same assassin anymore either. In the beginning, it always was a female Khajiit, but now I've met an Argonian and other races too. So it's still cool. Only too bad I can't kill that Astrid, nobody should send assassins after my nice character!