By Captain Veloth and his guards near the gatehouse only. When I FT in by the earth stone and walked in that side it was fine.

Bump. Now any saves I had after me returning to solstheim after beating the DB main quests must be corrupt cause I'll load and random NPCs will attack me for no reason.