Haha. The exact same thing happened to me.
I think what occurred was that I was dressed as a member of the powder gangers when I entered the casino.
The town of Prim has had a rough time with Powder Gangers, and won't appreciate seeing one in close proximity.
Try to change clothes.