Since I have the ability to start a new character, no I don't need to move my Perks around.
I like this.
Perks mark my character's growth. If I took them away, it would mean she forgot everything she learned - definitely not what I'm going for. It's fine to try something new, but I don't want to negate my previous experiences.
It's not that I dislike the idea - we've all chosen some perks that turned out to be completely useless and then wished to take them back, haven't we? - but I wouldn't use it. If I want to play a mage, I'll retire the thief and create a mage.