So did Torygg betray Torygg when he accepted Ulfric's challenge to duel?
It's hard to justify that because on one hand they are of the same people and country, so they share the same ideals and practices to an extent. But on the other hand they are of two different warring factions. So there's a lot of grey area, because they are kind of on the same side but not. Civil wars are tricky; in the American civil war there was brothers fighting each other on ether sides.
Roggvir knew Nord tradition, and was acting based on that. But the rules have changed, Ulfric is an enemy. And the Imperial army is in charge of solitude, and the Nords are not. If the Imperials weren't meddling with the Nords he probably wouldn't have been executed.
Yes, he was a traitor to the Imperials and their war.
No, he wasn't a traitor to the Nords and their way of life.