I let him live, because he quotes http://www.imperial-library.info/content/many-headed-talos, a piece of work that USED to be considered semi-lore, written by former dev and contracted writer, Michael Kirkbride.
Whenever I do the companions quest line and turn into a werewolf for the first time I make a bead line towards that Battleborn that wants to bash my face in.
I kill Heimskr the first chance I can. He's a moron. I also kill every person who annoys me. If they won't die, then I loot their houses etc repeatedly.
Heimskr and Cicero are the two most annoying peole to me. Can't believe that they aren't akilled by any npcs
Not Cicero he is my favourite Ho ho ho, he he he, break that lute across my knee, and if the bard should choose to fight, why then I'll set his clothes alight!