After level 50, you need to level more skills than before to level up, although the time to level up each skill grows gradually.
This is made to avoid characters having to much perks, because if the leveling system kept the same during the entire game, the max level would be over 100.
The amount of 'experience' needed per character level up goes up by 25 points per level. The math is pretty simple: 25 times the current character level, plus 75 (so going to level 2 requires 100 xp, level 3 requires 125, 49 requires 1300, 50 requires 1325 etc). Each time you increase any skill, you get the new skill level in 'XP'. So increasing destruction from 70 to 71 gives 71 points for character level up (I think).
So at higher character levels, it's not so much that you need more skill increases to gain a perk, you just need to be working skills at higher levels. Regardless of character level, bring up a skill from 15 to 20 is about as much progress to the next character level as bringing one skill from 99 to 100.