Large Heightmap, TESAnnwyn, L3DT, Creation Kit

Post » Sat Nov 17, 2012 1:14 am

Hello all, I'm part of a team attempting a total conversion of Skyrim into Westeros.

One our talented members has created a heightmap of Westeros. It's 4096 x 10240. I'd like to keep it to scale for now. I have searched and read quite a few threads on this topic but these have not entirely solved my problem.

These are some things I know:
"bUseCharacterRB=0" This line reverts the havok behavior system back to an older iteration that does not have problems with worldspaces larger then 4 quads.
TESAnnwyn converts heightmaps into esps that can be loaded into the CK, in theory.
L3DT is a powerful terrain editor that can also import heightmaps.

Hoddminir's blog is great


My workflow:

I had a psd file that was 8 bit grayscale.
I converted it to 16 bit grayscale, raw, no header, IBM order (?, that endian business that I don't quite understand)
I kept the resolution to 4096 x 10240

I used TESAnnwyn 0.23beta first. The following line are the option I set:
TESAnnwyn -i Skyrim -w Westeros -p 1 -b 16 -d 4096x10240 -x -64 -y -64 -h -8192 WesterosMap4096.raw


I could not load this esp at all. The CK would CTD.
I scaled down the resolution to 2048 x 5120, prepared an esp as above but changing the res values.
The CK did load this esp, but in the preview pane and on the heightmap editor, I saw endless peaks and voids.

I used L3DT Pro 12.06, loaded the original 4096 file as the heightmap, when I looked at it in 3D mode, the same peaks, but no voids.
I clipped the max and min heights to 500 m and -100 m, but the result was the same.
I made sure I was using a horizontal scale of 1.8288221359252928


What am I missing? Could someone troubleshoot this?
I'd like to keep this as one large worldspace, but I haven't been able to import a usable esp or esm.

I'm willing to provide links to files if that would help.

I think it might be the raw image I'm using, but I don't know how to fix it.

My PC is decent: ASUS P8P67 Pro, i7 2600k, Corsair 8GB, EVGA 570, a boot SSD, lots of space on a storage HDD.

I'm at my wit's end and need help.
User avatar
Blessed DIVA
 
Posts: 3408
Joined: Thu Jul 13, 2006 12:09 am

Post » Sat Nov 17, 2012 11:49 am

A few things that come to mind:
- Is TESAnnwyn reporting over-/underflows?
- Try using the scale argument for TESAnnwyn (-s x.xxx). Lower it until you don't get over- and underflows.
- Since you reference the Hoddminir blog I assume you are aware of having to limit the range of the grayscale?
- Have you tried making the heightmap a square with dimensions that are multiples of 1024 (e.g. 16384x16384)? Just expand the canvas size on the current heightmap in Photoshop. I don't know how picky TESAnnwyn is when it comes to heightmap dimensions. You can limit how far the player and the camera in the map view can go in CK.

I'd like to point out that you gain nothing by converting your 8-bit heightmap to a 16-bit heightmap unless you intend to do more work on it to add more detail. The conversion process won't magically make the gradients smoother. Ideally the heightmap would be a 16-bit file throughout the process or at the very least from the point that you start adding details to it and onward.
User avatar
Alina loves Alexandra
 
Posts: 3456
Joined: Mon Jan 01, 2007 7:55 pm

Post » Sat Nov 17, 2012 5:42 am

These are some things I know:
"bUseCharacterRB=0" This line reverts the havok behavior system back to an older iteration that does not have problems with worldspaces larger then 4 quads...
The havok bug only affects the x axis, so you won't need to add this. If your x axis size is over 4096, then you need to add this line for your mod to work correctly. Y axis is unaffected.
I used TESAnnwyn 0.23beta first. The following line are the option I set:
TESAnnwyn -i Skyrim -w Westeros -p 1 -b 16 -d 4096x10240 -x -64 -y -64 -h -8192 WesterosMap4096.raw
You should have used
TESAnnwyn -i Skyrim -w Westeros -p 1 -b 16 -d 4096x10240 -x -64 -y -160 -h -8192 WesterosMap4096.raw
...I could not load this esp at all. The CK would CTD...
It's too big to load as an esp. You would have to change it to an esm, but you would also have to alter the FormID (using TESVsnip) of your worldspace first, so that it begins with 01 (or 02 if you are including Update.esm as a master). This will also mean that you will have thousands of errors when you first load the esm, but the CK will fix them all. I just place some heavy object on the Enter key and leave it. On a map that size, it could take over an hour, but it will eventually fix them all. Once fixed, save straight away.
I scaled down the resolution to 2048 x 5120, prepared an esp as above but changing the res values.
The CK did load this esp, but in the preview pane and on the heightmap editor, I saw endless peaks and voids.

I used L3DT Pro 12.06, loaded the original 4096 file as the heightmap, when I looked at it in 3D mode, the same peaks, but no voids.
I clipped the max and min heights to 500 m and -100 m, but the result was the same.
I made sure I was using a horizontal scale of 1.8288221359252928

This happens when you forget to export the raw with a manual scale.
You must remember that when you export your raw file, you must change the setting to export as a manual scale raw and set the manual scale to 0.114301

If you still have problems, PM me a link to the file.
User avatar
Kevan Olson
 
Posts: 3402
Joined: Tue Oct 16, 2007 1:09 am

Post » Sat Nov 17, 2012 11:17 am

Ok, wow, I hadn't considered any of these things!
Thank you so much guys, I will try these out and report back.
User avatar
Michael Russ
 
Posts: 3380
Joined: Thu Jul 05, 2007 3:33 am

Post » Sat Nov 17, 2012 5:18 am

A few things that come to mind: - Is TESAnnwyn reporting over-/underflows? - Try using the scale argument for TESAnnwyn (-s x.xxx). Lower it until you don't get over- and underflows. - Since you reference the Hoddminir blog I assume you are aware of having to limit the range of the grayscale? - Have you tried making the heightmap a square with dimensions that are multiples of 1024 (e.g. 16384x16384)? Just expand the canvas size on the current heightmap in Photoshop. I don't know how picky TESAnnwyn is when it comes to heightmap dimensions. You can limit how far the player and the camera in the map view can go in CK. I'd like to point out that you gain nothing by converting your 8-bit heightmap to a 16-bit heightmap unless you intend to do more work on it to add more detail. The conversion process won't magically make the gradients smoother. Ideally the heightmap would be a 16-bit file throughout the process or at the very least from the point that you start adding details to it and onward.

I do get a ton of over and underflows. My question is: if I change the -s option in TESAnnwyn, won't that reduce the size of my map with respect to how many cells it covers? Or does that only affect the z coordinate, as in how high or low the mountains and sea levels are?

I limited the grayscale so that the sea level was 100 to 99% and the highest peak was about 13%. I did this through the Levels.. option in photoshop.
Wait, I think I might have misunderstood this part a bit. My sea level is at 100 but my highest peak is at 13. I think this may be the problem. Will report back.

I tried changing the proportions to a square but that did not help at all.
User avatar
Wayland Neace
 
Posts: 3430
Joined: Sat Aug 11, 2007 9:01 am

Post » Sat Nov 17, 2012 2:49 am

If you still get over and underflows, or cellborder breaks after you correct the grayscale range I can take a look at the heightmap if you like.
User avatar
Carolyne Bolt
 
Posts: 3401
Joined: Mon Jul 10, 2006 4:56 am

Post » Sat Nov 17, 2012 7:16 am

TESAnnwyn's scale argument only affects the height of the terrain. If I remember elinen's tutorial correctly, the heightmap should only have grayscale values that span from 76 to 100 (with 100 being black). Once you have the grayscale values limited correctly you can start tinkering with the scale argument to adjust the terrain height differences to your liking and/or try to fix remaining issues.
User avatar
Phillip Hamilton
 
Posts: 3457
Joined: Wed Oct 10, 2007 3:07 pm

Post » Sat Nov 17, 2012 11:49 am

OK, I shopped the file, grayscale is now correct between 76-100. I found that with the following options...

TESAnnwyn -i Skyrim -w Westeros -p 1 -b 16 -d 4096x10240 -x -2048 -y -5120 -h -2048 -s 0.05 WesterosMap_Flat4096-1.raw

I get the following result from TESAnnwyn...

Some gradient overflows/underflows have been caught and blocked:
Total Overflows: 1
Total Underflows: -2
Highest point is 490 THU (3920 Game Units) = 55.860000 metres [Cell (-2016,-5027)]
Lowest point is -256 THU (-2048 Game Units) = -29.184000 metres [Cell (-1921,-5120)]

Total number of cells imported: 40960

Finished! The imported ESP is called tesannwyn.esp and is 128 cells by 320 cells.

I try opening the esmified file, an dI still get the same stuff.
I want to scream! I'm going to step away from this a couple of hours and post a link to the files.
User avatar
Leticia Hernandez
 
Posts: 3426
Joined: Tue Oct 23, 2007 9:46 am

Post » Sat Nov 17, 2012 8:25 am

Trying to link on my tenth post.

https://www.dropbox.com/sh/m6yctfplytmeaxq/ejUe___kMl

It should have a psd and a raw file


Edit: the link works!
Anyone who can help me is welcome to take a look. What I would love is a protocol that worked, so that I may learn how it's done.
Edit 2: The following link has the original files from the team's cartographer extraordinaire.

https://www.dropbox.com/sh/014dj2r4xp8h05t/gUDOZrIvlw
User avatar
Naomi Lastname
 
Posts: 3390
Joined: Mon Sep 25, 2006 9:21 am

Post » Sat Nov 17, 2012 8:32 am

I got it earlier.
Trying to load the esp into the CK right now. It's taking a while, but hasn't crashed yet.
User avatar
Shannon Lockwood
 
Posts: 3373
Joined: Wed Aug 08, 2007 12:38 pm

Post » Sat Nov 17, 2012 5:23 pm

I got it earlier.
Trying to load the esp into the CK right now. It's taking a while, but hasn't crashed yet.

Thanks for looking at it.
I'm sure its the raw image, it has a lot of detail, I think that's whats giving me all the cell breaks. I'm almost at the point where I'm thinking about using threshold to just get a flat 2d map of land versus water and then try to use a program to rebuild the terrain. This will be my last resort scenario. It's a ton of work that I want to avoid if possible.

I've found even more forum posts on this. It's amazing how the same usernames keep popping up again and again. Maegfaer, Elinen, that guy who's creating Mesogea, I forgot his username... Pioneers of the process.
User avatar
Natasha Callaghan
 
Posts: 3523
Joined: Sat Dec 09, 2006 7:44 pm

Post » Sat Nov 17, 2012 4:27 pm

Couldn't get it to load in the CK, it's just too big. For a map this size, you'd need a cut down Skyrim.esm, to be able to load it. That's what the folk above use for the large worldspaces.

Anyway, I've resized it to 3072 x 6144, so I'm going to give it another try. Will report back soon.
User avatar
Sebrina Johnstone
 
Posts: 3456
Joined: Sat Jun 24, 2006 12:58 pm

Post » Sat Nov 17, 2012 3:24 pm

All done !!! :banana:

See PM for a link to your esp.
User avatar
Anna Beattie
 
Posts: 3512
Joined: Sat Nov 11, 2006 4:59 am

Post » Sat Nov 17, 2012 8:52 am

Hello all, I'm part of a team attempting a total conversion of Skyrim into Westeros.

One our talented members has created a heightmap of Westeros. It's 4096 x 10240. I'd like to keep it to scale for now. I have searched and read quite a few threads on this topic but these have not entirely solved my problem.

These are some things I know:
"bUseCharacterRB=0" This line reverts the havok behavior system back to an older iteration that does not have problems with worldspaces larger then 4 quads.
TESAnnwyn converts heightmaps into esps that can be loaded into the CK, in theory.
L3DT is a powerful terrain editor that can also import heightmaps.

Hoddminir's blog is great


My workflow:

I had a psd file that was 8 bit grayscale.
I converted it to 16 bit grayscale, raw, no header, IBM order (?, that endian business that I don't quite understand)
I kept the resolution to 4096 x 10240

I used TESAnnwyn 0.23beta first. The following line are the option I set:
TESAnnwyn -i Skyrim -w Westeros -p 1 -b 16 -d 4096x10240 -x -64 -y -64 -h -8192 WesterosMap4096.raw


I could not load this esp at all. The CK would CTD.
I scaled down the resolution to 2048 x 5120, prepared an esp as above but changing the res values.
The CK did load this esp, but in the preview pane and on the heightmap editor, I saw endless peaks and voids.

I used L3DT Pro 12.06, loaded the original 4096 file as the heightmap, when I looked at it in 3D mode, the same peaks, but no voids.
I clipped the max and min heights to 500 m and -100 m, but the result was the same.
I made sure I was using a horizontal scale of 1.8288221359252928


What am I missing? Could someone troubleshoot this?
I'd like to keep this as one large worldspace, but I haven't been able to import a usable esp or esm.

I'm willing to provide links to files if that would help.

I think it might be the raw image I'm using, but I don't know how to fix it.

My PC is decent: ASUS P8P67 Pro, i7 2600k, Corsair 8GB, EVGA 570, a boot SSD, lots of space on a storage HDD.

I'm at my wit's end and need help.


I'm pretty lazy, so I havent read the whole thread, but just the starting topic.

Your problem is all about the right scaling of grey tones.

Plus, for what I can read, you used 8 bits images, instead of 16 bits. BAAAAAD!!
16 bits have a wider range of values that can be used to define altitudes!!! It should be your standard.
Plus!!! You told TESAnnwyn to interpret your heightmap as a 16 bits when it's just a 8 bits one (-b 16)!!!!!

And now write down this scaling: max altitude with 16 bits heightmaps is 7471 meters (or 7466)!!!!
This means you have to set your output heightmap to have that max altitude so that the grey tones will get scaled based on that value.
To make it clearer:
- a 16 bits heightmap, full black-to-white ranged, defines altitudes between 0 and 7471 meters (underwater included)
- if you dont set your L3DT output correctly, your tallest mountain's peak will be full white, which means 7471 meters tall!!!!!!
- if your heightmap has grey tones differences between two contiguous pixels that define altitude changes higher than ~14 meters, you'll have tears in the terrain meshes!!!

That's why I prefer to use World Machine 2.2, because it makes this all much easier and, to be honest, I think it's also perfect for this purpose.

Anyway, if you want, I opened a thread about the height scale for L3DT output heightmaps on L3DT's forums like....few weeks ago and got the info on how to do it. Just search there.
User avatar
Carys
 
Posts: 3369
Joined: Wed Aug 23, 2006 11:15 pm

Post » Sat Nov 17, 2012 3:27 am

I generate LOD for Westeros just to make sure everything turn out correct and found that
you may have some problems with the Westeros.LOD file that Oscape generates.
I'll post back with a fix (hopefully).

[img]http://homepage.ntlworld.com/tambos.turf/Skyrim/Westeros_LOD.jpg[/img]
User avatar
Scotties Hottie
 
Posts: 3406
Joined: Thu Jun 08, 2006 1:40 am

Post » Sat Nov 17, 2012 5:18 am

I generate LOD for Westeros just to make sure everything turn out correct and found that
you may have some problems with the Westeros.LOD file that Oscape generates.
I'll post back with a fix (hopefully).

[img]http://homepage.ntlworld.com/tambos.turf/Skyrim/Westeros_LOD.jpg[/img]

I always suggest using http://skyrim.nexusmods.com/mods/10896, to generate "worldspace.lod" files, because it's very simple to use and works perfectly.
User avatar
Natalie J Webster
 
Posts: 3488
Joined: Tue Jul 25, 2006 1:35 pm

Post » Sat Nov 17, 2012 4:16 am

I always suggest using http://skyrim.nexusmods.com/mods/10896, to generate "worldspace.lod" files, because it's very simple to use and works perfectly.
That worked a treat.
KatoPlato, for your world LOD, generate it with Oscape and download the program that Zehryo mentions. When running the program to make the lodsettings file, use these as answers

1. Westeros
2. -64
3. -96
4. 192

Copy the Westeros.LOD file to Data\lodsettings\
User avatar
Tamara Primo
 
Posts: 3483
Joined: Fri Jul 28, 2006 7:15 am

Post » Sat Nov 17, 2012 3:29 am

THANK YOU SO MUCH!!

I was having the hardest time figuring out where those problems were originating from. Now, I understand where I was going wrong and more importantly how to fix it, I have a working esp, and my next problem might be solved already!

Honestly guys, I don't know how to adequately express my gratitude.

Thank you!
User avatar
natalie mccormick
 
Posts: 3415
Joined: Fri Aug 18, 2006 8:36 am


Return to V - Skyrim