it's a feature, not a bug! Devs added it to either motivate you to fight against them or with them depending on your likes.
i like fithing with vampires, the problem comes when NPC get killed by them (specially vendors). That's one of the reasons why people hate that feature.
please fix it bethesda, don't like visiting ghost towns or cities.