Can you please teach me how to force 2x SSAA, would really like to see how it looks like in the game, did you just select 2x AA and then 2x Supersampling?. Ive already force AO via inspector and it looks nice.
If you want to see something great try AO. You need Nvidia inspector, load skyrim's profile with it and choose 0x00000003 (Fallout 3) as Ambient Occlusion compatibility. Then under [common] enable AO usage and choose Quality as setting. It's a BIG hit to performance, still get a SS and compare it with a non-AO SS.
Now about TrSSAA, I am not sure that it even works
AFTER having made several more tests at White Run. I didn't have time to test Transparency AA in a more appropriate place, but I guess that results won't be much different. I've tested at 2560x1600, taking SS and then comparing to a different settings SS, at 200% zoom. Here's in short what I've discovered:
Driver based MSAA works on items (e.g. Dragon Plate armor) and even 2x (at least at 2560x1600) is enough to correct the jagged edges, but it doesn't work on environment (e.g. stairs). Game based MSAA works on the environment and you better use 4x as minimum (if one can afford it ofc), while it doesn't work on items (like inventory items and don;t know what else). I am 100% sure about this, I have tested extensively. Still, I don't know which or how many objects fall under each one of these 2 categories (inv item being the one and environment items being the other). For me, playing at 2560x1600 with 1.5gb video cards, the best option is 4x MSAA from game and 2x MSAA from driver. NO TrMSAA from driver (it doesn't give any benefit according to my tests and it can screw game based AA), but enabled TrMSAA (set value to 1) from game (although I didn't test if that actually improved anything, still my rig takes no hit from it).
To do this,
you need nvidia inspector to remove the "treat override as application controlled" flag, or you are not forcing anything. If you have available VRAM, you can play with AA settings a bit and see what works best for you. At 2560x1600, the best for me is 4x MSAA from game and 2x MSAA from driver. Enabled TrMSAA from game, disabled at driver.
---------------------------------------------
Now, if we exclude the fact that I can't go to really high AA settings because I am VRAM limited at 2560x1600 (which wouldn't have much visual difference tbh, since even without AA I have to look at the SS at 200% zoom to realize the jagged edges),
the only setting that actually provides a performance hit to my rig is the all mighty Ambient Occlusion, but the outcome is definitely worth it. Without AO, I could probably be locked at 60fps even with one 580 (with a bit higher o/c). With Ambient Occlusion even as high as "Quality", I am locked at 60fps. But it's close due to cards running at 87% to 99%, usually at 97%. With "High Quality", which is the max AO setting, I get 41fps. Fortunately, at the spot I tested, "Quality" actually looked better to me, among all settings. So imo, even a single GTX580 setup (coupled with a capable CPU and with a conservative o/c like mine) should be fine without AO, even up to 2560x1600, but for AO (which is the single setting that does the greatest difference, at least according to my taste) you need a 580 SLI setup. About FXAA... I don't like. It does a great job with jagged edges, but only because it provides a blurred image. At least for 2560x1600, that trade off is definitely not worth it.
i7 750 - 4.0 Ghz (o/c)
GTX 580 SLI - 860 Mhz (o/c) 1.5gb
vertex 3 (ssd) - Dell u3011 2560x1600.
P.S. TrMSAA = Transparency Multi Sample AA, don't mix this up with MSAA (the non transparency one). Except from the settings I describe above, I also have 16x anisotropy forced from driver and also enabled in game + INI optimizations for best visual quality (e.g. 4096 shadow map, no object fade, tree/land shadows, etc).