I have found that Alt+Tabbing or restarting the game will restore me to 30-60 FPS depending on location... but it's really ruining immersion, and the game occasionally crashes when Alt+Tabbing. I've searched and it seems a few other people had the same problems, but most of those had no resolution, or it didn't apply to me. My specs are:
AMD Phenom 9750 (x4) 2.4Ghz
Geforce 560ti (1gb) / 290.53 driver
8 Gb DDR2 RAM
Vista x64 Home Premium
I have tried: Skyboost, TESVAL, FPS Booster V1.1 - which seem to help my top-end FPS, but don't stop the major drops. Overclocking my video card a ton (no change,) Of course ENB9 Anti-Freeze (part of STEP recommendations.) And even Low settings without Mods don't significantly help the problem (maybe a little.) I also tried disabling a few cores, setting process priority to High. Probably tried other things too that I've forgotten... have been trying to fix this since mid-December and it's making me pull my hair out. It might be my imagination, but it seems to be getting worse... originally it was mostly when exiting places like Dragonsreach or Labyrianthian, and now it seems to slow when exiting about anything.
Since it will run any area at 30-60 FPS after Alt+Tabbing, I really don't think it's my hardware. I do have parts to build a PC with a slightly better processor, but that would mean disabling the current PC, installing and reconfiguring Windows and Skyrim... and I'm not at all sure it will fix the problem.
I would be soooo grateful if anyone could help me figure out what is wrong.

