i recently unlocked the matching set bonus and i found that it had no effect on my switching the daedric helmet for a dragon priest mask.

Does this mean dragon priest masks are considered to be daedric? or is this just a bug

+i heard theres a limit to how much armor rating you can have, 567 or something like that, truth?

they're either light or heavy armor

As far as I know, Dragon Priest masks are internally classified as Daedric armor, even though they can have either a Light or Heavy designation. As such, the Daedric Smithing perk improves their upgradeability, and they are affected by the Matching Set perk when wearing Daedric Armor. Granted, I'm speaking with a Ph.D. in wiki-reading, so take all of that with a grain of salt.

All the dragon priest masks, including the wood mask, have the ArmorMaterialDaedric keyword and match Daedric armor for purposes of matching set.

The armor cap is 667, for an 80% damage reduction, but there is a hidden 25 point bonus for each piece of armor in the four body slots (head, torso, feet, hands). So if you are wearing a full set of armor, and your visible armor rating is 567, you are at the cap. If you only have three pieces of armor, you'd need a visible rating of 582. I believe that the bonus only applies to light and heavy armor, not clothing or jewelry.