Just trying to figure out if I'm doing something wrong. I'm currently level 40 and I'm trying to finish my build except that I feel like I'm short on perks. I keep having to raise other skills I don't really use in order to level so I can get another perk to then place in a primary skill. Am I doing something wrong here?

Bringing a skill from 0 to 100 will net you ten perks, but only two skills can be maxed out with ten points, so it's quite possible you'll need to raise other skills in order to get all of the perks you want in a primary.

And of course, skills don't start at zero, so even Light Armor and Smithing won't generate enough perks on their own to max themselves out. It's true that you can use a skill extensively and not need to max it out, my current warrior will likely end up selecting every block and heavy armor skill, but only put a few into smithing and one handed weapons. However I still think it's entirely reasonable that you need to level up a few secondary skills in order to get all of the perks you want for your primaries. Books are good for this, as are the occasional quest rewards that boost skills (Hermaeus Mora's quest in particular).

some secondary skill perks are very useful.

pickpocketing allows you to carry 100 more lbs.

archery alows you to zoom in.

look farther up the perk tree to see what perks work for you.

Let me see.. short on perks..

Did you spend a ton of perks on crafting?

I'm curious about how leveling works, the fine details. If you max every skill you'll reach level 81.5, there are 18 skills (1800 points) and characters start with 305 points (25+5*20+12*15), so there are 1495 skill increases available to a character during a game. As you reach a higher level it takes more skill increases to reach the next level and increasing a skill from 90 to 91 seems to count more towards a level increase than increasing it from 20 to 21. Are there formulae anywhere that explain how skill rank affects its contribution towards the next level (for the sake of discussion lets call the contribution XP) and how the XP requirement increases with character level?

Yes. I went up the right side of smithing up to daedric and then up the middle of enchanting. But I still had the same issue where I needed to level something else to get me up some levels for perk points.

Right, because you used about 10-12 levels for your smithing/enchant perks.
Since the soft cap is around 50 (only people... with too much time actually take characters to 81) that is 20% of your perks, and since you are only 40 it is 25%. Your class is part blacksmith/artificer, since you went perk heavy in 2 crafting trees.
You aren't doing anything wrong, but crafting is part of your "build" since you invested so much into the perks, and perks determine a build in this game.

I have the opposite problem. At level 52 I have 8 perks and nothing to spend them on.