Admittedly, yes... five was too small a number, but apparently you didn't get the point I was making.
A damage bonus that is natural to High Elves and no one else was the point.
It's, iffy. Although it could be done, it's still not very convincing for a few reasons. There's not really much to say in the lore that Altmer get more damage out of Destruction magic. They are known as good mages overall, not only in Destruction. What kind of bonus would you give them for the other magic schools? It's much harder to devise such bonuses.
Also, please note that a small starting bonus of +5 to a skill at birth has actually a VERY strong effect in the corresponding skill. That's because untrained, someone with a +5 racial bonus in a skill starts the game with level 20. Which means that untrained in such skill, you can still put two perks or more in it than a race that starts with no bonus for that one. It does have a nice ring to it in that if I train my Orc mage in Destruction there won't be much at all for an Altmer mage to pretend to be much better than me, thus furthering the idea that training overcomes birth in the Elder Scrolls world. But without training at all in, say, Restoration, just by using two perks in it an Altmer will display much better capabilities in that field. +10 in a magical skills even means you start the game at Apprentice spell level already! If that's not significant at all, I'm not sure what is.
Once again, I'd like to restate the fact that the Elder Scrolls game world leans on the side of the fence that says that training trumps birth. You can go all angry at that notion, nothing good will come out of it because that view and the oposite and BOTH very valid ways to construct a game world. You cannot say one of "realistic" and the other and both ideas produce working and plausible game worlds. I'm happy to have some games try the "training about birth" vision for variety.
Also, the player character is the one that blurs the difference between races mostly which is perfectly justified because such differences aren't god imposed absolutes in the first place! They are laws of averages mostly : the orcs are on average stronger and better with weapons than altmers. But the player is as far from the average member of his specy you can get so he's allowed to be an Altmer stronger than any orc you can meet in Skyrim as long as you build him for that.