See http://www.gamesas.com/index.php?/topic/1295824-nvidia-29036-beta-drivers/ for original beta driver thread about AO, as not all get good results from the latest beta driver.

Note this is applicable to using the latest nVidia beta drivers with AO support added to TESIV profile.


AO only looks its best when other settings are in place to maximize the quality too. This includes eliminating some shimmer effects. Remember if you have trees accepting shadows this can be a big source of perceived issue, when in reality it's the junky blocky shadows of the game itself that is doing this in the 1st place. I use that setting because I like it giving the trees a bit more depth so I will reference those settings too.

On my rig I did extensive testing in the woods outside Riften for the hit and results of quite a few variations. The final settings I settled on giving max quality and still having stellar 60FPS is the following:
(note the hit I see on the steps of Whiterun outside Dragonreach is minimal compared to normal).

In SkyrimPrefs:
[Display]
bFXAAEnabled=0
bTransparencyMultisampling=0
iMultiSample=8
MaxAnisotropy=16
bTreesReceiveShadows=1 <-- Source of some perceived shimmering due to blocky edge shadows, in particular flashing across leaves. That's the games shadow junk not AO fault.
bDrawLandShadows=1
fShadowDistance=6000 <- Default is 4000. Can crush a rig if huge number. 6K pushes them out to a point I notice fade-in much less while still excellent performance.
fShadowBiasScale=0.25
iShadowFilter=4

Load nVidia Control Panel:
Select the Elder Scrolls 5: Skyrim profile.
Antialiasing - Mode = Enhance the application setting
Antialiasing - Setting = 2x (having nVCP match the game 8x will produce huge extra load for no gain. Just let the game do 8x since it actually works nice)
Antialiasing - Transparency = 8x (supersample) <-- very important for total quality.
Maximum pre-rendered frames = 0 (<-- max reduction in input lag if your system can swing it. If it can't, slowly increase this until you reach lag / performance balance)
Apply the settings and close nVCP.


Load nVidiaInspector:
Load the Elder Scrolls 5: Skyrim profile
Under Common change only this:
Ambient Occlusion setting: High Quality.
Click Apply Changes and close nVI.

Open nVCP to verify and you should see Ambient Occlusion now says Custom.

Done. Close it. Go test it.

I experimented for about an hour with a very large variation of settings on / off / increase / decrease values and the above is the best, for me, in quality and performance.

My results are now excellent shadow depth on wide range of everything, including swaths of grass that now have a "thickness" to them due to the AO affects.

It may seem to dark for some but I'd rather have it dark with thickness than without it in my own comparisons.

Edited:
Added section [Display] for clarity
Added 2 extra shadow related settings I have that aren't defaults.

Thanks for the tips. I'm anxious to try these out tonight.
I'm running the 290.36 driver which allows the AO settings to be made in the nVCP. Is that sufficient or does nVidiaInspector do something better?
Also, what does setting AA to Enhance 2x in the nVCP do when you have AA set to 8x in the game?

One more question. Which of these settings actually gets rid of the shimmering with AO? I had AO enabled last night and the shimmering was beyond unacceptable.

You'll need to use nVI so you can change it to High Quality AO. No idea why nVidia chose only to provide Performance and Quality settings without the HQ option too.

I'm not a technical expert on DirectX however there are plenty roaming around that could answer that in extreme detail.

Using 2xFSAA in nVCP prevents it from trying to double-apply it since the game is already doing it as far as I can determine. The other option would be to set it on App Controlled but that may or may not disable the transparency 8xSS being applied regardless of what it says afterward. That's the real meat of the change in the nVCP. To get the 8xSS trans to apply. Experts in the area may be able to explain it with precision. I only know that during my testing, having it also at 8xFSAA in nVCP uber loaded the boards and dropped performance, leading me to conclude it was a double-application of the feature when it's not needed, so dropping it to 2x in nVCP removed that issue entirely while retaining exceptional 8xFSAA through the games settings.

From my own testing though, it's the total combination of 8xFSAA (so many edges, espeically in the woods, critical for it), 8xSS transparency (no idea how many objects in the world truly apply here but it's important) and HQAO (QAO wasn't enough) it entirely eliminated tree top shimmering as I roam the woods. It also eliminates the shimmering of the grass, leaving thick patches with deep shadows and no shimmer at all.

The only other shimmering left on the trees is very minimum and looking closely can be determined it's the trees getting shaodws cast on them, and of course the game shadows are ultra blocky edged, and therefore when trees sway the blocky edges flash across the leaves give a "flashing" effect.

Edited:
Fixed typo, added more info.

Don't have NVidia Inspector, but it looks like AO works and works beautifully. I'm running on a 560 Ti though - so I'm still taking some FPS hits, but nothing too serious. I'll have to see if I get this shimmering problem people seem to have.

Thanks McCleud. I'll try the AO setting via nVI. I tried the Quality AO settings and there was too much shimmering.

Was roaming the file and see 2 shadow related settings I have that are not default so I added them incase it has any impact in this area. Check post but they are:
fShadowBiasScale=0.25
iShadowFilter=4

Well, I tried running with AO HQ settings last night and it's a no go for my tastes. Still way too much ugly shadow halo/shimmering , especially around bushes. Bummer.

Quick, random question: how long does it usually take for a NVIDIA driver to go from BETA to official release status?

McCleud, I'm wondering what type of rig is necessary to run this. Also, are you running a bunch of other tweaks/texture packs, or are these the only modifications you have made? I'm excited to try these out but don't want to get my hopes up too high, after previously hearing that even quality AO has a large performance hit.

Last question, how does this AO work with fog? I had heard somewhere that the combination of the two was problematic, but this may only apply in Quality mode..

My rig (which I shall return to come holidays) has an i5 2500k and 2 560 ti's in sli. These shouldn't match your 480's but I'm wondering whether they will still suffice...

I'm running 2x 470 in SLI (i7 950 @ 4ghz). With the OP settings I get 40fps, but I like to keep closer to 60 - it's just smoother gameplay for me at least. Anything above "performance" AO is too much fps loss for the subtle effect. The 8x Transparency SS is too much for me also so I keep it 2x or disabled - really can't see the difference anyway. I think there's something wrong with AA as I have 8x AA in the ini and 2x AA in nVidia CP - still jaggies.

I've been testing. On my rig having AA set to Enhance 2x in the Nvcp while having 8x Sparse Grid SS on greatly increases FPS. This leads me to believe the Super sampling is not working while Enhanced 2x AA is on. Setting AA to application controlled but keeping SGSS on 8x is a massive hit to fps.

Will test further.

Edit: Because isn't the AA setting in Nvcp Multisampling AA which is not compatible with the Super Sampling AA at the same time in the control panel?

Edit 2:

Good tip though about the Ambient Occlusion being set to High Quality. That fixes the bad looking grass with the Beta Driver Skyrim setting which was the only thing bad about it I noticed.

FXAA doesn't have a quality setting like "2x", "4x", etc., it's either on or it's off.

I suggest people try disabling the main AA in Skyrim, Multisampling, and enable FXAA under "Advanced" and see how it looks with 8x Super-Sampling Anti-Aliasing Transparency. For me, this looks even better than using 8x Multisampling AA in Skyrim while freeing you up to do even more tweaking in both the INI, addition of HD textures, and Nvidia Inspector. Most people's rigs won't be able to do all both 8xAA in Skyrim AND 8x Super-Sampling AA Transparency through Nvidia Inspector while being able to tweak other stuff, and the quality you might gain is nothing compared to other tweaks. This is from roughly 20 hours of tweaking since Skyrim was released. I can give you all my recommendations if you wish, but I'm not using the latest Beta drivers due to other problems they can cause, instead using 285.79 where you can still use SSAO fine.

Note: AF is different than AA and should be left on for best quality.

My rig:
Core i7 920 @ 3.5 GHz
12GB DDR3 RAM
2x GTX 470 Overclocked in SLI
2560x1600 resolution

Note that if you use a resolution lower than 1080p you may need additional Anti-Aliasing to get similar quality, which is why it's always good to get 1920x1080 when you're at or near high-end specs.

Skyrim isn't shown under my program settings tab even with the 290.36 drivers. When I go to add program and select it, it says that AO isn't supported. Do you know a fix for this?

Skyrim is in the 290 drivers, search for "e" letter.

"Elder Scrolls 5: Skyrim"

This is the official title for the game (technically "The Elder Scrolls V: Skyrim")

I checked for that too. I was just didn't feel like typing all of that out.

I followed the nVidia Tweaks guide for Skyrim too. Though I HAD to use nvidiaInspector in order to change anything. nVidia's Control Panel just would NOT find Skyrim on my PC. I'd have to manually add the TESV.exe and even then it didn't seem to work right. I'm running AO @ High (not HQ) @ 1920x1080 with "Ultra" settings (everything within Skyrim maxed). When out and about in the forest I'm showing a fairly consistant 60FPS with both GPUs @ 99%.

We have very similar rigs - do you have Transparency Multisampling or Supersampling enabled? How about AA? Curious what you have your ini set to and the nvidia control panel.

Un-check the box that says "show only programs on this computer." Then you get a list with a million different programs and it will be listed in the E's, this is how I had to add it.

Not sure if it was the only settings in total related to getting it as I see / saw it at the time. I also use larger memory pools but I don't even know if those bytes related settings actually work.

With the 480's I sustained 60fps with the settings shown, except in towns of course but we all know that is due to other issues.

Changing the distance of shadows can have a very substantial impact on performance even on my rig at the time so definitely try it at 4k default and see if improvement in speed.

SS transparency is important to reduce the shimmering junk. HQAO won't change that, it'll only make the AO itself better.

In most games it is usually possible using nVI to apply extra settings such as Sparse Grid which can be awesome visually but I messed with it for awhile during my testing at the time and it always produced problems where an apparent shadow layer, like another off screen buffer for them, wound up on the screen at the same time. Very horrendous.

If you have the correct driver version and it doesn't detect you can just Add then browse to the binary and select it, and nVCP will then match the binary to the one stored in the profile and show it as Elder Scrolls 5: Skyrim and you're good to go at that point.

just did some testing with this and the new driver...yeck...

The OA for skyrim looks bad in a fog situation.

The OA setting for Oblivion on the other hand looks better, is faster and thats what i am sticking with for now...The new driver does have better overall visuals compared to the Driver i was using...266.666

The new driver with all settings max in nvidia inspector sure does warm up my card...

My usual settings at 40 % fan usage 40-50 FPS 98 % use on card. netted me 50 c Even after 15 hour session.

2xSS
4xAA
16x AF

Cranking things up a Tad

8x Super sample
32xS AA
16x AF

Really warmed my card up....60% fan usage 30-40 FPS 95 % use of card and went to 80 c (but looked really good!!)

I can really hear that Fan...lol Will be keeping my Original settings with Oblivion compatibility for OA.