And then I got rid of the shadows using this http://skyrim.nexusmods.com/downloads/file.php?id=16464 and I went from 20 FPS with mods to 60 solid FPS with the same mods.

Does this mean my CPU is to blame for my crappy FPS?

i7-875k @2.93 GHz

If so, how can I fix it? Or maybe it is just the way Skyrim is coded... or do shadows run off the GPU?

Shadows in Skyrim being rendered by the CPU is a myth. Shadows are rendered by the GPU. And always have been.

Your CPU is more than fast enough for Skyrim.
Skyrim was CPU-intensive when it was released. Because it was shipped without basic compiler optimizations. This has been fixed many months ago. People still keep saying Skyrim is CPU-bound. But it isn't. It's like many other games. The GPU is still the most important piece of hardware in your PC for Skyrim framerates.

If removing shadows is improving your fps, that is an indication your GPU was the limiting factor. As it should be. Note, there is no surprise that if you remove shadows from the game, that your fps increases. That is to be expected. Shadows are compute-intensive. They require hard work by the GPU. If you remove that work, stuff will speed up. What did you expect ?

There is nothing to be fixed. If you want higher fps, buy a faster videocard. Easy.

If you want to know how your hardware is doing, download a tool. GPU-Z or nvidia inspector or MSI Afterburner. Then use it to see your GPU usage, see your vram usage. Use the windows Task Manager to look at your CPU and memory usage. Then you get the exact numbers, and you won't need to speculate.

Okay, glad to know the shadows are GPU bound. I am planning on buying a GTX 680 pretty soon. Just am hesitant on getting a 2gb card because I have a feeling it will be obsolete in the near future (I could be extremely wrong about that though).