I've used DDSopt on my Architecture and Clutter folder so far, and they've shrunk quite a bit (Architecture was cut down by about 60 MBs, and Clutter is about 16MB smaller). I don't know how well it'll work in-game yet or what all I might have broken, though. I figure I'll also process the Dungeon, Furniture, Landscape, Plants, Sky, Terrain, and maybe ImpactDecals and Effects (not sure on the last two) before trying anything in-game with the newly-compressed textures. But I basically just set DDSopt to "Skyrim" mode, pointed it at a folder and just clicked Process, so we'll see what happens. Hopefully I don't break anything and don't lose much in visual quality in the process.

At least I backed up my entire Textures folder before starting this. lol

Well, for now no knowledge of the alphas of Skyrim's textures has transpired. So in a worst case it could happen that all texture go to hell. I just wrote the tool, I didn't disassembled shaders/effects and hack-up NIFs to determine what's the alpha inside a "_d" or "_bl" for example. But if I wouldn't have nobody would start looking really what's those alphas ... well there is no guarantee it'll start now, but I hope so.

I ended up not doing Plants, ImpactDecals, Effects, or Terrain (I figured it would likely break something or not make much of a difference anyways). so far, I haven't seen anything not working correctly in Whiterun or Solitude, and things are generally running better. I'm not getting more FPS, but my framerate is a lot more stable (as in the FPS isn't dropping to 0 or something for several seconds when the VRAM gets overloaded, which happened a lot in Whiterun; my mod list was just too beefy for my rig, but I haven't gotten around to stripping out the performance-killers yet). Load times in certain areas seem like they've improved somewhat, too.

I didn't measure VRAM usage or anything like that, but things *do* seem like they're running better so far, and I haven't seen any borked textures yet.

Hey, that's nice. Maybe there really still is a shader-switch and alpha-blending has been "optimized" away by using DXT1.

Yeah, the key benefits will be reduced or completely negated stuttering (in relation to VRAM overload, there can be other causes of stuttering of course) and thus a more consistent framerate, and reduced loading times. And naturally, having VRAM to spare allows you to run more texture mods, or turn on more features like AA or SSAO etc.

Also, Ethatron's tool has another advantage -- it creates more detailed mipmaps, so textures from near to far will look better for everything.

Once he's sorted out the remaining issues with how DDSOpt chooses to compress textures, I'm basically going to run it on my entire Data dir (all original Skyrim .bsa textures, and all installed texture mods) and run it on all newly downloaded texture mods

Of course, texture mod authors could also run it on their mods before they package and upload, and specify in the description or README that the mod has already been DDSOpt-ed.

DDS are actually marked (in the header). It's possible to identify already optimized DDSs from DDSopt when you don't process them further.

You have to be careful though when it comes to normal maps and compression. I've used the DXT5 compression which screwed up and pixelated my mipmaps. I ran some tests with custom mipmaps, not auto generated, and I got the same issues with the DXT5 compression so I was forced to use more memory in order to get rid of the pixelation. I saved them in the 8.8.8.8 format and my problem was solved however the file size was twice as big;

http://static.skyrimnexus.com/downloads/images/4141-1-1324691093.jpg

http://static.skyrimnexus.com/downloads/images/4141-1-1324851134.jpg

Sorry to be a pain / knob, but could someone post a link to the DDSopt download. I have a few of these "naughty" / unoptimised retex mods - already unzipped in a separate folder, which I coudls apply this to proir to rezipping and reinstalling.
I was looking through the DDSopt thread but couldn't find the link (plus can't get on Skyrim nexus from work now - NetSense has reclassified it as "advlt/Mature Content" now - hope its not due to me d'loading 80m worth of mods when I was bored!)

Yeah, this was covered. What you can try to do is resize the normalmap to 1/4 the original size (half width, half height) and use uncompressed 8.8.8.8. This will produce the same file size as the DXT5 version, and generally look better due to having no compression artifacts, despite being lower res.

Another thing you can try to do what Ethatron suggested, using partial derivatives to store the normals, which should help hide the DXT artifacts and possibly increase precision in some situations. I don't really know how to go about doing that, though.

It's http://www.skyrimnexus.com/downloads/file.php?id=5755.


DDSopt has a setting in the menu where you can enable that for tangent-space normal-maps. It's on by default. http://obge.paradice-insight.us/wiki/Modder%27s_Manual again. I made it in a way that "normalize(tex.xyz)" yields the correct vector. It's compatible with any engine which normalizes in the shader (which should be every). For object-space normal-maps this can't be done because you'd have singularities and it matters to have full freedom or range for z.

If you want to see exceptional cases of the adaptive max. derivative which is stored in z, then look at the http://www.skyrimnexus.com/downloads/file.php?id=4834 normal maps in 4x4 cell size. They look weird but's because no application I know of normalizes normal-maps on display, just load it into photoshop and look into the green channel. The steeper the maximum normal in a 4x4 block the darker, that's when the partial derivative get's scaled down by that factor. It's not perfectly linearly interpolatable in the mathematical sense, but's still better than normal normal-maps, which are way less linearly interpolatable, and it doesn't artifact.

A good question is what to do with normal-maps with black and white matte. I really ask myself what the engine does/produces when it interpolates black and a valid normal-vector ... It also totally screws the DXT palette selection.

Are you talking about normal maps that have black in the unused gutter space? I never thought about this.

Yes, there are some object-space normal-maps with tangent-space forward vectors as matte [0,0,1], some with black [-1,-1,-1] and there are some tangent-space maps with white as matte [1,1,1]. All of them have gradients towards that matte on the borders. It's impossible to get the normal-vector rotation towards those values out of the interpolation. If one of those pixels is hit by a shader and not masked out by the UV, it'll make a big mess.

Other question, does Skyrim support CxV8U8? I'll be happy to convert object-space normals to that format.

Just to clarify: I think the concept of "normalize(tex.xyz) yields the correct vector" should work for object space normals.

What will not work, for object space normals, is the specific implementation trick which relies on x and y always being smaller than z. The algorithm would have to be generalized for object space normals (and I imagine you would need special treatment for 4x4 patches where the largest coordinate varies from pixel to pixel -- perhaps even two special treatments, one for smooth patches and one for sharp patches?)

I'm no expert...but for the moment I'm going to pretend I am. Feel free to correct me if I'm wrong

Textures shouldn't even be using alpha channels unless you have layers (which, as far as I know, Skyrim textures don't support layers) as alpha channels are for transparency.

Of course I'm not counting the "other" textures such as normal maps (which you need the alpha layer to determine the shinyness) Speaking of, does anyone have a list of all the extensions (as in _n)?

There can be transparent parts of a texture even if it doesn't have layers.

EDIT: ((It gets kinda confusing when you start your sentence with "They're" but then rewrite the sentence and it should start with "there" but you already typed "they're" so you get befuddled...))

Haha, I knew what you meant

But yeah I did not think of that. However people shouldn't be saving textures with an alpha channel just because (as I have seen in the past).

Agreed! I'm pretty sure that's mentioned in the OP, and if it's not, it definitely should be mentioned.

  if (stristr(path, "\\menus"))	return (!ch ? "User-interface element" : "Masking");  if (stristr(path, "\\faces"))	return (!ch ? "Aging-map for faces" : "None");  if (stristr(path, "\\facemods"))	return (!ch ? "Aging/tone-map for faces" : "None");  /* normal maps */  if (issuf(path, "_n") ||	  issuf(path, "_fn") ||	  issuf(path, "_xyz") ||	  issuf(path, "_xyzd")) {	if (stristr(path, "landscapelod") ||	stristr(path, "terrain"))	  return (!ch ? "World-space normal-map for terrain" : "None");	return (!ch ? "Tangent-space normal-map" : "Specularity-map");  }  /* normal maps */  if (issuf(path, "_msn"))	return (!ch ? "Model-space normal-map" : "None");  /* glow maps */  if (issuf(path, "_g") ||	  issuf(path, "_glow") ||	  issuf(path, "_emit"))	return (!ch ? "Glow-color" : "Glow-mask");  /* hair gloss maps (l/rgb-only, greyscale) */  if (issuf(path, "_hh"))	return (!ch ? "Gloss-map for hair" : "None");  /* hair detail maps (rgba) */  if (issuf(path, "_hl"))	return (!ch ? "Detail-map for hair" : "Opacity");  /* metallic masks */  if (issuf(path, "_m"))	return (!ch ? "Reflectivity-map for light-sources (greyscale)" : "None");  /* environment masks */  if (issuf(path, "_em") ||	  issuf(path, "_envmap") ||	  issuf(path, "_envmapmask"))	return (!ch ? "Reflectivity-map for environment-maps (greyscale)" : "None");  /* environment maps */  if (issuf(path, "_e"))	return (!ch ? "Environment-map" : "None");  /* shine maps */  if (issuf(path, "_s"))	return (!ch ? "Specularity-map for skin" : "None");  /* skin maps */  if (issuf(path, "_sk"))	return (!ch ? "Tone-map for flesh/skin" : "None");  /* backlist maps */  if (issuf(path, "_b") ||	  issuf(path, "_bl"))	return (!ch ? "Backlight-map for fur/ice" : "None");  /* parallax maps */  if (issuf(path, "_p"))	return (!ch ? "Parallax-map (greyscale)" : "None");  /* diffuse maps */  if (issuf(path, "_d"))	return (!ch ? "Diffuse-color" : "Opacity");  /* haze maps */  if (issuf(path, "_h"))	return (!ch ? "Haze-pattern" : "Unknown (please report)");  return (!ch ? "Color-map" : "Opacity or parallax-shift");

Updated the OP with the following to draw attention to Ethatron's tool:


The Solution Ver 2 (kinda)
``````````````````````````````
Ethatron has released his DDSOpt tool which does a great job at picking up and fixing (by re-compressing a texture or a normal map correctly) some of the issues discussed above. It also has an excellent mipmap generator that adds higher levels of details in the mipmaps, leading to even better texture quality at mid to long ranges.

So -- for users users and especially texture authors -- you can run DDSOpt to optimise textures to be correctly saved in the right format, and thereby reduce VRAM usage. It is still in development however, so if encounter any odd results please post in the http://www.gamesas.com/topic/1330264-ddsopt-of-use/.


-----> However, there is a large caveat <-----

DDSopt can correctly re-compress textures and can sometimes pick up minor issues such as unnecessary alpha channels, but it is not a silver bullet to magically make all your textures perfect. It cannot fix errors in textures and alpha channels created (however inadventently) by the texture author. Things like left-over borders, stray pixels in alpha channels and a whole bunch of stuff that actual texture artists could probably mention that I can't.

In other words, if you make and upload texture mods DDSOpt can help you ensure your textures are properly saved with the most optimal DXT compression format, but it cannot fix problems with textures, alpha channels and normal map generation. This is up to you to get right first time before you upload the mod -- and unfortunately, while testing DDSOpt with Ethatron, there are a lot of mods that have these sorts of issues and as a result DDSOpt can't optimise them. So if you're a texture author, please check your work

DDSOpt can be http://www.skyrimnexus.com/downloads/file.php?id=5755.

Awesome! Thanks

what is

ch

in that code fragment about file names?

Yep. Not so bad to be understood I hope. Was to lazy to make a propper HTML-list.

I added now an extreme good algorithm to utilize the available buckets in the 565 format for model-space normal-maps. That is 1:2 compression (from RGBA, on 2:3 from RGB), I think it's acceptable quality. At least better then DXT.

So, um, what does this mean in laymans terms?

EDIT: Also, check your PMs