I think you missed my point...

I do think maybe if it just locked the points to a tree... so if you find out you don't like a perk you can dump it into another perk in the same constelation... that would be cool IMO. I could get a lot more out of my characters I think. Maybe pay for 1 perk at a time with "loaned" skill pts, earn a (current) lvl worth (or maybe lvl per perk tier) of the attributed skill before you continue leveling up said skill. This would represent the time needed to retrain your skill for the new perk, and you can't retrain another perk in the same constellation until you've finished retraining the last, and of course you can't break perk requirements either.
To move from light armor smithing to heavy armor smithing, would be a lengthy time investment... but I could see people doing it. Plate & elven weapons is pretty nice for example, but you could use those perks later when you get smithing up to use all ebony, and never waste any perks.