saying "start over" is hardly worth posting.
I think you misunderstood. Im saying quit to desktop and run skyrim again. Not start over by creating a new character.(Or am I also misunderstanding what you meant by "start over")
As for the problem, if it is indeed that you finished the quest "No One Escapes Cidhna Mine" it may be that is the cause of the bug. Either reload from earlier save when you just got out from Cidnha mine and stay back and let the forsworn do their thing. Hide if you will while the onslaught happens.
Edit: Sorry not Forsworn Conspiracy quest.