No but the loss of an Enchant slot does. Offense is king in Skyrim.
You get the slot back if you wear a circlet, which my current character does in lieu of a helmet. Yeah, he won't get the set bonus, but since he's going Heavy (currently in Orcish, eventually will wear Dragonplate) he doesn't need it.
Remember, if you are not wearing a helmet, the armor cap goes up to 592 because of the hidden 25 you get just by putting on a helmet. You can still reach the cap without a helmet, but if you stop at 567 and are not wearing a helmet, you are not at the cap.
EDIT: @bobjim123. I heard it was 25 for each piece, but you could be right about it being 40 for armor and 20 for each of the other pieces.
The cap does not increase if you add a helmet, as it's based on the mitigation cap of 80%. Your armor rating will increase, but it wont actually matter.
Mitigation% = TotalArmorRating*0.12
TotalArmorRating = HiddenArmorRating+DisplayedArmorRating
HiddenArmorRating = 25*NumPieces; shields don't count, so max of 4
80% = TAR*0.12
80/0.12 = TAR
I'll multiply by 100/100 for clarity, so 80/0.12 becomes 8000/12
8000/12 = 667
667-100 = 567
So, while you'll have a 592 AR, the added 25 aren't actually doing anything unless the other guy has both a mace and the armor bypass perk, and that's pretty damn rare.