I've noticed that most of the targets are in crowded areas, and never sleep, making it nearly impossible to kill them without getting a bounty. Fortunately, I have the misdirection perk, which lets me slip them poisons without incurring a bounty.
So, I actually came up with a rather sadistic means of killing them: Lingering Damage Health! Imp Stool and Orange Dartwing are the two most common ingredients with this effect. The duration for custom potions is always ten seconds (I spit at that, and I wish I could make it longer). With 100 alchemy skill, all five "Alchemist" perks, and 100% fortify alchemy enchantments, I can create a potion with these ingredients that is precisely 5 magnitude. This means that the targets will endure 50 health damage - just enough to kill them, as they have exactly 50 health - except that it's done over the course of ten seconds!
Imagine the sadisticness! I'm not satisying with simply running in, clobbering them with a warhammer, and getting out of their before anyone has time to blink! No, their death has to be a lingering, slow, suffering one!
The only thing better is if I could increase the duration of my potions! I'd make a potion that only does one point of damage, but over the course of 50 or more seconds. I'd like to drag their suffering out for as long as possible. Can somebody make a PC mod that allows you do that?