Every time I go into the Castle all the vampires act like there is an enemy around but there Isn't and it won't let me talk to Harkon. Anyone know how to fix this!

Dawnguard outside i think in the tower to the left when you first Fast Travel to The Fort of Undead

THanks