Isn't it kinda logical that's it dangerous near the houses, I mean it's frickin' Skyrim.
You're never safe in Skyrim.