Hi there Stormshout! Good to see you.
There seems to be a flaw in your main point. As of now, Perks are purely a game mechanic. You do not learn them, you do not have to go up to a trainer and learn the ability, you are not required to complete a quest. Infact, the only requirement is to level up, and that can happen over you exerting some other random skill. So, it is highly inaccurate to describe perks as learned knowledge. Hence, the act of reallocating perks is as a game mechanic as perks themselves, and thus quite logical.
Not just a mechanic, it is representations of learned skills and abilities, learned over time. Picking them is a shortcut for the player, in the same way as paying for training, press key, go up skill, don't have to watch your character firing at a practice target for hours. It's the same with waiting, sleeping, jail time; just because you don't have to sit in front of a screen with a snoring character displayed for eight hours doesn't mean your character hasn't slept in their virtual world.
And who are you to tell us what kills or doesn't kill replay value? Skyrim is supposed to be a "do what you want" game - so if someone were to want to respec then they shouldn't be faulted for it. If the feature existed, and you didn't like it, no one is forcing you to use it. Derp.
Well, and only replying because you used the word 'derp':
By that logic, Paarth coming back to life, the sun coming back on, and saying 'I have all the shouts I need, I want the dragons to stop appearing now' are all perfectly valid. Just stick the magic re-perk stone next to the race change and 'I don't need all that stamina now, I'll take it as health' stone, and everyone who thinks "I don't want any adverse effects from my mistakes" is more important than taking a stab at decent rpg design is happy.
Oh, yeah, 'don't like it, don't buy it' is a terrible argument, about as valid as 'don't like it, don't use it'.