I have just found that I have a 'Justiciar Execution Order' for my character. Apparently talkng to one of them is enough to get this.

Anybody know how to clear it, or if its even possible (short of a save before the event).

Kill them first?

Kill them all.
I got one too, and all I did was talk to one or something as well.

From what I understand they have a "purge" but never record it, where they "purge" a village. And they can "arrest" anyone for just about anything and get away with it.

I talk to them all the time. There are no survivors.

They are Thalmor. Nazi Elves. Death at the tip of your blade/axe/mace/arrow/flamespell is better than they deserve.

And yes... "Talking" to them also works. The best tongue to "talk" to them in is Dovah.

Killing spree it is then

Well, you by your very nature as Dragonborn are a threat to their plans... so returning the favor is obligatory.