Skill leveling isn't an issue, as it goes by use. Perks, on the other hand, will be limited now, since there's only 81 which can be assigned. Considering your high overall level, you've maybe 10 perks left (getting to level 81 is difficult and tedious). This is maybe 5 per tree, depending on how you divide them.
If you're not concerned with the perks in the magic trees, you can easily accomplish 100 in both levels. It just means quite a bit more game playing.