So what happened is I used seduction on the priest before he died, completed my quest, we went into the castle he read the elder scroll and as he reads it the entire castle attacks him. Luckily this didn't bug anything. So I go to Soul Cairn, come back and everyone atatcks him on sight, I have to reload a save until he's secluded run up to him and use seduction on him a second time to get my forgotten vale quest.

Hmm, out of curiosity, if you change into a vampire lord in the castle, do they attack you?

It sounds almost like you may have a loose script overwriting something from dawnguard.

That's a good question. I'll check it out in a few minutes and report back.