As some of you know there is a large problem with making LOD files for trees. What you get is all the tree LODs that should be in cells beyond -32 to +32 line up at the border of 32/31 cell as a huge wall of trees.
http://imageshack.us/photo/my-images/221/walloftreesu.jpg/

A huge problem if you are making a world that covers an area beyond cell -32 or +32 in X or Y. Also a problem if you are making new tree LODs of Skyrim as some areas of the map is beyond this point as well.
The idea with this thread is to find a solution to this problem. Possibly with an end result in a program that can make tree LODs that work.
This is a very frustrating problem and yes I too feel that it could have been avoided with a better working CK, but we can either [censored] and moan about it, or do something to fix it. And frankly, complaining will get us nowhere. Try to minimize the ranting and instead be constructive so we together can solve the issue

What we need (in broad strokes)
* To find out what is in a BTT file and what functions the values have. (under way)
* How to get the position of trees from an esp/esm.
* Make a program that takes tree position information and writes BTT files.

I have found a nice spot in Skyrim to do some testing, cell 32, -10, from the tower of Mistwatch I have a nice view over the Eastmarch valley. From here you can see the 32,-8 and 32,-4 cell blocks nicely.

Me and Geggakloss have been working on this for a day now trying to see what we can find out.
The Creation kit makes two files for tree LOD, a .BTT for every 4x4 cell group and a LST file for the worldspace. From some http://www.gamesas.com/topic/1354877-new-worldspaces-object-lod-and-region-generation/page__st__30 it looks like the BTT file contains the coordinates for each tree, and the LST file defines what lod image should show for that tree.

Finding out what information is in a BTT file is a lot like solving a crossword puzzle in a language you don't know and it is all in little endian hex. Sounds like fun? Well it actually is
We did a lot of testing in new worldspaces, adding one tree at a time. So far this is what we got:

What is in a BTT?
The information in a BTT looks something like this:
(two Aspen01 trees and one Pine01 in the same cell)
02 00 00 00 00 00 00 00 01 00 00 00 00 20 09 46
00 e0 08 46 00 80 a7 c5 00 80 14 40 00 00 80 3f
70 0d 00 01 ac fb e6 2f 00 00 00 00 01 00 00 00
02 00 00 00 00 20 0a 46 00 c0 1e 46 00 80 a7 c5
00 40 7d 40 00 00 80 3f 6f 0d 00 01 ac fb e6 2f
00 00 00 00 00 80 1f 46 00 40 20 46 00 e0 a8 c5
00 40 75 40 00 00 80 3f 6e 0d 00 01 ac fb e6 2f
00 00 00 00

This is abit hard to read, so let me structure it up
Now this is what we think we know right now, remember this is guessing mostly.
02 00 00 00 (1)
00 00 00 00 (2)
01 00 00 00 (3)
00 20 09 46 00 e0 08 46 00 80 a7 c5 00 80 14 40 00 00 80 3f 70 0d 00 01 ac fb e6 2f 00 00 00 00 (4)
01 00 00 00 (2)
02 00 00 00 (3)
00 20 0a 46 00 c0 1e 46 00 80 a7 c5 00 40 7d 40 00 00 80 3f 6f 0d 00 01 ac fb e6 2f 00 00 00 00 (4)
00 80 1f 46 00 40 20 46 00 e0 a8 c5 00 40 75 40 00 00 80 3f 6e 0d 00 01 ac fb e6 2f 00 00 00 00 (4)

1) How many types of trees in total. (quite sure of this)
2) Treetype group for one cell, if trees are in different cells a new 00 00 00 00 is made for that cell. An existing group number can change when you add new trees. The numbers are not always in sequence, can be 01, 04, 05. Not really sure how this works, maybe related to the groups in the LST file or some indicator of what cell the group is in?
3) Number of trees of this type, in this case the first group is a Pine01, the secound group two Aspen01. (quite sure of this)
4) The values for each individual tree. (quite sure of this)

The tree information:
___X-cord__ ___Y-cord__ ___Z-cord__ ____Unknow01_____ __Uniq tree ID___ ____Uk02___ ___Uk03____
00 20 09 46 00 e0 08 46 00 80 a7 c5 00 80 14 40 00 00 80 3f 70 0d 00 01 ac fb e6 2f 00 00 00 00

X and Y – It looks like the X and Y coordinates are specific for a cell, not for the 4x4 cell block. What information tells what cell in the 4x4 block the tree is in?

Unique tree ID - Quite sure of this, but don't know if it is all these values.

Unknown01 – varies between each lod making run for the same trees with no changes made, some kind of time stamp perhaps?

Unknown02 – ac fb is constant between files made by two different CKs, second part varies between saves of the esp and between different files. The Original filles all have 00 00 00 00 here.

Unknown03 – so far always 00 00 00 00, but this is not the case in original tamriel BTT files.

Differences between CK and original BTT files
BTT for 28,-4 Original
87 4a e6 47 1a 84 74 c6 e2 09 20 c6 f2 98 a1 40 14 ae 87 3f 83 08 0b 00 00 00 00 00 00 00 00 00
BTT for 28,-4 CK made
00 40 e6 47 00 80 74 c6 00 00 20 c6 00 a0 a0 40 00 a0 87 3f 83 08 0b 00 ac fb f5 2f 00 00 00 00
First values (which is last in litle endian) for X, Y and Z are always 00 in CK made BTT. Coordinates don't completely match even within the 32 cell border where LODs work, but they are only a small number off. Could be that the LODs appear a litle off from the tree.

What is it that goes wrong in the CKs BTT files of tamriel compared to files that work?
BTT for 32,-4 Original
8d 6c 02 48 b1 37 97 c5 14 42 16 c6 5b 2b a4 40 00 00 80 3f 9f c6 03 00 00 00 00 00 00 00 00 00
BTT for 32,-4 CK made
00 e0 ff 47 00 40 97 c5 00 40 16 c6 00 40 f9 3e 00 00 80 3f 9f c6 03 00 ac fb 44 33 00 00 00 00

The X coordinates all get set to 00 e0 ff 47. I can manually in the hexeditor change it to the X coordinates from the original BTT file, and voila, the tree LOD moves out into the cell beyond the 31/32 border.
http://imageshack.us/photo/my-images/856/movetrees.jpg/
What this tells us is what we already know, the CK made a mistake, but the engine can handle tree lods in cell 32 and beyond. It also says something about these unknowns that vary between the original and the CK made BTT file, they are not necessary for functional lods... at least not in this specific case.

We will continue to test more, but it will take time to finish this project as we have very little idea of what it is we are doing
I invite you all to participate, all skills are needed!

A great initiative! I am very glad that the CK's generator is indeed at fault, and not the engine. I am sure eventually the community will manage to make a proper tree lod generator.

I'll try to help with this in the near future and do tests of my own and compare the hex values with yours. I'm far from an expert though, but I'll do my best.

Good work I'll take a look at the code when I can, if only to confirm.

This is a good thread, working towards a fix. This is probably one of the most important issues as any meaningful landscape changes etc cant be done without a fix for this.

Took a quick look at the BTT generator code - It does look like it's clamping some FP values. More testing's required but it ought to be fixable.

Will you marry me?

Woohoo!

Is it possible that we would see such a fix in the CKE?

Hmm, must consult the ol' ball and chain first. Not to worry though, I shall keep you posted.

Almost definitely. But it might be sometime before I even start working on it. Given the fact that we're still in patching season and I'm juggling too many projects at the moment, I really can't afford to invest any more time on Skyrim/the CK than I already have. Heck, I'm still sitting on the awesome-huge update the CSE's due

That said, depending on the circumstances, I might be able to whip up a very quick plugin to fix just this issue. We'll see how it goes.

That would be greatly appreciated!

So far I worked around it by replacing Trees with Static equivalents to make them part of the Object LOD, but this only seems to work as long as there are no "real" objects as LOD. Right now I can either get trees in the Object LOD or objects, not both at the same time. It's kinda holding us back, it's hard to judge a region with large parts of the LOD missing. With a fix we could go full steam ahead again!

I have the same problem with Mesogea. I havent had much time with a full time job, family and an hons degree to do so this week was essay time, the next three weeks are dissertation time, then I can get back to proper work (Mesogea). However I could have done a bit of something if the tree LOD was up to scratch and the trees were not so limited anyway. I am hoping ztree will have his fantastic trees ready soon and the tree LOD issue can be fixed then I can really start pushing ahead with region editing. At the moment all I can do is keep re-texturing some of the Skyrim objects to fit Mesogea.

@shadeMe
Thank for looking into this, much appreciated! Really good news that this could be fixable.
Geggakloss says he wouldn't mind looking into the code if you are to swamped.

Is it possible that the bug only affects .esp files? I had the wall of tree bug in my esp file, after i merged it to its master file via VC the tree lod renders correctly as far as i can say.

edit: negative/ after minor changes the problem occurs again

Is there a plugin file that I could use to generate BTTs with the CK?

Create a new worldspace, doesn't matter if it is empty. Dump a bunch of trees in it and you can generate them.

You might also need to convert the tamrieltree.dds to tamrieltreelod.tga (you can use nconvert for that) and place it in Steam\steamapps\common\Source\TGATextures\terrain\tamriel\trees. Or else the generator might complain.

I could also give you access to our MERP svn to download our latest build if you want, but that's a lot more data than you need for this.

Glad to see this is still being looked at.

This is great news- Ethas looks pretty bare without tree LOD. Any more news on progress?

Tree LOD's might not be needed for smaller worlds see my post http://www.gamesas.com/topic/1375308-experiment-with-is-full-lod-checked-on-all-objects-and-trees-with-good-results/

Unfortunately, with as large as our world is, that simply isn't an option.

I don't know, I keep adding trees and objects with no FPS drop. I realize not everyone has a computer as beefed up as mine though. I will eventually test on my other lower end computer to see the difference.
I would like to see other modders experiment like me and see what they get.

Is there any news on this?

I have not worked on this for some time. Today I have started to look into the x,y,z position cordinated in the BTT file, and how they relate to the position in game units that is show in the creation kit. I don't have anything substantial yet. I have gone from completly confused to at least seeing some kind of pattern. It looks like the BTT file has cordinated down to the 0.0001 game unit or less in presision.

Glad to hear of some more progress, as it's certainly a disappointing oversight in LOD generation.

After a lot of head scratching, we had a breakthrough this evening. Now I think we have figured out how to calculate the x, y, z coordinates in the BTT file from the X, Y and Z position of the tree in Game Units, at least for positive coordinates. Its a small but important step.
There has been a lot of testing and looking at the binary numbers for the coordinates given in hex in the BTT file before a pattern emerged
Next step is to put a tree somewhere beyond the +/-32 cells, and see if we can get a LOD tree in the right place. I'll post more details after we done some testing. But that will have to wait until tomorrow, have to sleep. I will definitely dream of 1 and 0...

Anyone know of/have an idea for a code that can get the X, Y and Z position of the tree in Game Units out of an esp/esm, so we can make the process automatic?

Beyond console/Papyrus commands, I can't give any suggestions. Also, have you found anything related to rotation/scale?

Congratulations, though, it's great to see progress emerge out of what seems like endless numbers.

The BTT coordinate system has at least three parts; Prefix range, whole numbers and decimals.
The Prefix range is divided in groups with 2-base game units, for example 2-3, 4-7, 8-15, 16-31, 32-63 and so on. The second part, the whole numbers are added to the lowest number of the prefix range.

Example
This tree stands on X coordinates 235983.6875 (Game Units). In the original tamriel BTT file, the hex coordinates are ec73 6648 (this should be read as 4866 73ec). And when we convert the hexadecimal to binary we get: 01001000011001100111001111101100
The binary code divides like this:
Unknown: 0100
Prefix: 10000
Whole numbers: 11001100111001111
Decimal: 101100

How to calculate the game unit coordinates from the binary numbers:

• To get the relevant range from the prefix n, calculate 2^(n+1). In this example the prefix binary 10000 is 16 in decimal, which gives us: 2^(16+1) = 131072.
  
• Convert the whole number binary 11001100111001111 to decimal and you get 104911.
  
• Add these two numbers together to get the first part of the game coordinate: 131072 + 104911 = 235983
  
• In the decimals of the coordinates every byte represents 10000/64 = 156.25 decimal units. Convert the binary to decimal, 101100 = 44, multiply with the decimal units 156.25 to get the decimal of the coordinate: 44 x 156.25 = 6875

We have not looked at how negative coordinates are written jet, so more on that later.