Yeah, as the title says. Anything that is applied via item can just be reiquiped but race/perk buffs are gone forever (or so it seems.) unsure about perks in general...
I wondered why my character who had maxed out shock resists (from enchanting) was taking so much damage from a mages shock spell. Need to check that out.