I'm using patch 1.5 and was still using nvidia driver 285.62 back from october 2010. I switched to the most current one 296.10 and I actually get less frame. I went from maybe 55-60 even outdoors to I don't know what number but significantly enough to tell the movement and framerate is a lot lower. And so I switched back to 285.62. Just wondering if this is the case for some people.

My video card is geforce GTX 560 Ti if that matters.

Nope. Each subsequent release has only improved performance for me, with 290.53 being a drastic increase. 460m

Make sure that if you are using Ultra setings that your shadows are on High.

I have GeForce 9800 GTX, after installing 295.73 WHQL, the only issue I had was screen flickering every 30-90 seconds (mostly on desktop apps, but I havn't noticed that in Skyrim). Then I rolled back to 285.62, and now no screen flickering issue. I'm not quite sure, but I guess Skyrim ran a little faster with 295.73.

Yes my shadows are on High, and everything else is set to highest / max value.
I shouldn't be complaining, so what if an older driver actually runs my skyrim faster....lol