So when I'm in Markarth coming out of Understone Keep, or at the entrance to Markarth looking up at the inn... or outside Riverwood and looking at it or the entrance to Riften, or walking around Falkreath forest, etc. my FPS drops a bit, but goes away when I look away. Why is this?
Specs:

2 GTX 580 Classy Ultra

16gb RAM

i5 2500k @ 4.8ghz

1tb HDD

Windows 7 64-bit, latest drivers (beta graphics drivers)

Is the game poorly optimized still? I mean, the only real graphically intensive mods I'm using is HD 2k textures, W.A.T.E.R. (I think it's intensive anyway), Climates of Tamriel, Lush Flora, etc. and there may be more but I can't remember.
Please tell me it's not my system.

Games like Skyrim may have issues with performance in some areas when going "full on" with visual improvement mods. Without the mods you would almost certainly not get frame rate loss with that rig, its the mods that are the poorly optimized part here.

On top of that there are a whole plethora of issues that can arise with SLi systems. This is particulary true when using "beta" versions of drivers.

What I would do its test with just one card active and see how performance is. Then test with various settings on the Nvidia control panel with just one card to see whats the biggest frame hit with regards to AA performance. If its still a bit picky, try rolling back to a known "good" driver set for Skyrim and making sure you have an SLi profile set for the game itself.

It may also be worthwhile checking out MSi afterburner and its options for Nvidia cards in SLi.

Jobesky sees the same thing I do. The compiler optimizations introduced in patch 1.4/1.5 (similar to TESVAL and Skyboost) seem to be missing in 1.7.