Did you do the Cidhna Mine quest?
Yes. Which was a contributing factor to me wanting to kill the Silver-Bloods. Not to mention the fact that one of them is marked as essential until after its completion. However, the foreman and the guard wouldn't be upset over Cidhna Mine. They would be upset that I killed the Silver-Bloods, which they shouldn't even know about. None of the guards are calling me a murderer. Nobody else in town is, either. Only those two guys who worked for them. I didn't side with the people
in the mine, if that's what you were asking. After the quest, the Silver-Blood thanked me, and I shot him with an arrow in stealth mode.