This is a fix that worked fine for me and I saw it here earlier so it is not MY fix, I'm sorry to say I don't remember their name.

Anyways:

Disable in the options in the Skyrim launcher for the graphics, disable the AA and AF completely.

Go into your own graphic cards properties and use AA and AF via that instead of letting the game do it.

When I did that and set the shadows to high instead of ultra, I got such high FPS at times it was over 250FPS.

I had to put Vsync back on to even stabilize the FPS and since then I have a constant 55-60 FPS (Vsync limited, and the mouse is sort of OK too). Granted I haven't been to a big town yet, but it at least for me made a big difference.
This leads me to believe that they have a problem with the engine in handling AA and AF, also seems to be a problem with loading the high quality textures too.
Since at times you get REALLY low end textures on some objects whilst others are OK.

Then again the textures over all are really poor in quality and only good at distance.

At least it's something to try, it worked for me in regards to performance so far.