I fortify both types of armor with Health - it may be a wash between this and fortify armor...dunno - just how I do and have done...and it's totally worth it for me. So probably worth fortifying the armor too.
Combine the four items (necklace, ring, cuirass, boots) and Fortify Light/Heavy can be quite useful, indeed. At higher levels one may wish to switch for Fortify Health or Health Regen. Hope that helps