Do you kill any of your neighbors? I know in the Lakeview House, I always killed that Necromancer down the road via range from back porch of my house.

Windstad manor is not too far from Ustengrav; I found some draugr corpses on my front lawn. Rest assured, any that come to my house will be killed and their remains used to make trophies.