[REL] TESVsnip (Skyrim Edition) : V

» Sat May 26, 2012 10:27 pm

figments's TESsnip Fork (Skyrim Edition) version 4.2
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

TESVSnip is a fork of the TESsnip utility bundled with Fallout Mod Manager with changes specifically for Skyrim.

In the words of the original author, TESsnip is a low level plugin editor that allows you to open esm/esp files, view the record structure and cut/copy/paste records and subrecords between different parents, including different esp's. You can also view and edit the data saved in a subrecord, in either hex or string format, as well as editing record headers.

So I been making periodic edits recently to my local copy of TESsnip and maybe more than really required since there are more usable utilities like SkyEdit available now and tes5edit and the CK available soon. I thought I'd make a second release now that I have a first pass at most of the records in the skyrim.esm file. This should at least make it easier to make certain edits not possible in other tools at the moment.

Requirements:
.NET 3.5

Change Log:

Spoiler

v4.2
- Fix null pointer exception when editing a records with no subrecords
- Various recordstructure.xml updates
- Fix issue with duplicates in navigation history

v 4.1
- Fixed issues with MGEF (signed vs unsigned values)
- Added Copy & "Copy To" to search window
- Renamed Tools menu to Strings
- Edit Reorder Subrecords to work on multiple select or group or plugin
- Added several menu commands for manipulating strings
- Explicitly save string table files seperate from plugin save
- Export string table to Xml file
- Import string table from Xml file
- Import strings from master string tables
- Internalize strings
- Prune unused strings from table
- Move Internalized strings to string table

v 4.0
- Introduce "Batch Edit" on Search Window
- Introduce "Column Selector" when using Basic Search or Search with "Type"
- Add Favorite Searches to Basic Search dialog
- Add Find References to Tree Context menu and Ctrl+LeftMouse on report links
- Fixes to RecordStructure.xml from thirdparties

v 3.9
- Introduce "Basic Search" to the find dialog drop down
- Add basic ability to select elements to search on via Basic Search
- Auto display columns involved in Basic Search

v 3.8
- Fix subrecord parent reference when using Full Record Editor to avoid issues with report
- Map Numpad-*,/,+,- to ExpandAll, CollapseAll, ExpandSelection, CollapseSelection
- Map Ctrl+Numpad-*,/ to ExpandAllSelection, CollapseAllSelection
- Fix hyperlinks so that multimaster esp/esm links resolve correctly

v 3.7
- Change default answer for deleting items to Yes.
- Change main view ordering to fix status bar being above incremental find
- Alter "Copy To" to include parent groups when copying loose records like ACHR
- Add "Report..." to tree context menu for opening standalone report for selected record

v 3.6
- Fix quick links in "Add Master..."
- Ask when closing plugin if user is sure
- Add F6 keyboard hooks for switching between windows

v 3.5
- New docking windows manager with better usability. (Still some bugs in it)
- Fixed most performance issues with search (also capped at 1000 results via config)
- Swapped Ctrl+F and Ctrl+Shift+F so Ctrl+F opens search panel and Ctrl+Shift+F is incremental

v 3.4
- Add search results form (experimental - i.e. may have poor performance)
- force refresh of subrecord list when record selection is changed

v 3.3
- Miscellaneous fixes where subrecord parentage was lost when copy/paste of record. Fixes EDID refresh issues.
- Fix refresh of main text panel when subrecord changed
- Change location where docking windows are hooked to allow floating positions to be loaded on startup
- Remove global menu hook for Delete key and handle on a control by control basis

v 3.2
- Handle resize due to DPI scaling before main view is fully so constructed

v 3.1
- Extra error handling to try and track down errors
- Handle issue with use of Flag Dropdown combobox while control is still being initialized
- Fix with multiselect delete from key binding
- Allow Down key to open Flags Dropdown combo

v 3.0
- Major rewrite to use new TreeList control for Record List
- Include Sort and Drag/Drop on single TreeList to allow reordering
- Restructure tree controls to support multiple select operations
- Manually sort form id list rather than using native sort which lead to inconsistencies
- Remove check for VK_BACK during WM_CHAR as it is incorrect an interfers with VK_OEM_DOT
- Add modeless record Editor on context menu to allow side by side edits (experimental)

v 2.6
- Fix issue with the new FormID Editor not saving values
- Fix issue with the new Repeating Element Editor not saving values
- Fix issue with the Paste Node feature with Groups causing nodes to not be created as expected
- Fix issue with all filter on formid not showing all
- Sort records on formid editor
- Add Record Compression option. Disabled by default.

v 2.5
- Bug fix to avoid using shared memory when editing leading to incorrect values being saved

v 2.4
- Bug fix to so that repeating elements such as KWDA can be added in new control
- Add config only option to allow old subrecord editor to be used

v 2.3
- Use RTF5 control if installed
Greatly improves text rendering when links are involved and has better layout as well
- Docking windows for Record and Subrecord (not much new but can undock and relocate)
- Reformated the main display text for subrecords to use tables
- New Medium Level Record (looks like old but overhauled for future extensibility)

v 2.2
- Add toggle all button to the ESM load filter dialog
- Add expand/collapse options to tree context and edit menus
- Add option to Add Master to plugins from Edit menu or context when plugin selected
- Bug fixes for errors preventing subrecord from loading

v 2.1
- Fix bug where hex editor was not saving changes

v 2.0
- Change REFR display so X,Y are divided by 4096 to match cell locations
- Add "Form ID Ref" search type which is a backreference search for records using given formid
- Fix Conditionals on global variables display
- Fix progress bar update / cancel during search

v 1.10
- New language handling items to better support localization (Experimental)
- Better handling for Windows Copy/Paste mode
- New Icon - Scissors snipping at Skyrim is what its supposed to be

v 1.9
- Add Setting for MaxHistoryItems in config file and default to 30
- Fix History control so that redo is truncated when a new selection is made
- Add Disable Hyperlinks since some pages are very slow to load due to hyperlink processing
- Fix subrecord copy/paste so Ctrl+C and Ctrl+V work again
- Workaround issue with richtext box causing sound to be made when setting text
- Fix bug with search backward not actually searching backward
- Hacked workaround so that Del key mostly works on the find text field and is not handled by main menu

v 1.8
- Fix QUST after change to CONDITIONAL broke it

v 1.7
- Convert Text Search to a background incremental search
(Enables cancel during full text searches)
- Add Name w/Type and Full w/Type to allow for searches limited by record type
- Experimental short descriptions for REFR, ACHR showing location information
- Fix FirstTimeLoad of skyrim.esm where user selects cancel to options and filter is still applied
- Change main text field area to RichTextBox and add hyper links to items
- Add Forward/Back navigation buttons for the Record TreeNode

v 1.6
- Add option to use Windows Clipboard instead of internal copy
(This is experimental and may have issues especially when copying large chunks of data)

v 1.5
- Change copy/paste semantics so appended items are not reversed

v 1.4
- Add setting to allow specification of file open or file save locations
- Add Context menu on tree to allow copy to other plugins
- Update the Sanitize Order Spell
- Substructure matching probably slower but more flexible
- Added NAVI, CELL, REFR, WRLD, LAND, PHZD, PGRE, ACHR and SCEN
- Fixed form look ups for plugins

v 1.3
- Bug fixes for crashes and other misbehaviors
- Fixed positioning for first time loading
- Synchronized RecordStructure.XML with UESP wiki
- Fixed nonconforming record search

v 1.2
- Bug fix in handling lstring which prevent sub editor from opening
- Support multiple subrecord copy/paste (only from the subrecord toolbar)
- Change IDs for GMST and EFID

v 1.1
- Major update to RecordStructure to allow inline groups
- Programatically tried to discover layout of all records so we have a better first pass
- Improved usability on the subrecords by adding toolbar
- Improved incremental search and nonconforming record search
- Improved performance when doing a lot of activities like search or generation of detailed reports
- Added Ability to exclude certain record types on load of skyrim.esm to conserve memory
- Added Str4 data type for fixed 4-byte strings

v 1.0
- Add LString data type to represent localized string
- Import the strings directory if present
- Export strings to the Strings directory if strings are present for a plugin
- Add a simple string viewer for localized strings (hard coded to English)
- Update the LString data type for thinks like FULL subrecords

Notes:
- The check box on the LString record editor is for controlling whether to
store the integer ID (which is in Hex) or to store the actual text.
- When checked it stores the actual text. When not checked it stores the id.

TODO:
- QUST record is mess and may require a new XML format to deal with it properly
- Substructures like MO2T and MO3T have complicated repeating structures not possible to edit currently
- Sorry the new string editor got away from me and is rather fiddly and not intuitive but that is in line with TESsnip anyway
- Detect localization or allow selection of localization in future
- Add/update spell to convert from ESP to ESM which also fixes LString types

Credits:
* timeslip - The original author of TESsnip
* Dave Humphrey - Maintainer of http://www.uesp.net/wiki a source ESM layout
- Tes5Lib (http://github.com/uesp/tes5lib)

License:
* The TESsnip code is licensed under GPL so this is by extension also GPL
* Included third party components are covered by their own license

Source:
git://github.com/figment/tesvsnip.git

Downloads:
http://www.skyrimnexus.com/downloads/file.php?id=5064 [skyrimnexus.com]

» Sat May 26, 2012 4:14 pm

thank you, and happy new year!

» Sat May 26, 2012 6:00 pm

Thank you very much!

One comment though. When I resize parts of the GUI and then click anywhere, the components return to their default sizes.

Edit: Also, for the ammo records, YNAM is the sound made when picked up and ZNAM is the sound made when dropped.

Edit2: This might help you, if you haven't looked at it already: http://www.uesp.net/wiki/Tes5Mod:Mod_File_Format

» Sun May 27, 2012 7:40 am

Thanks. I'll take a try with it, much of my work's forced to be in tessnip due to the lack of flexibility in some of the other tools.

Edit: LVLD on the leveled item lists is recorded as a string, pulls an error when working with it. Just pointing it out. LVLO entries are recording a formID as an integer, too. Small corrections in the xml are fine though.

Edit2: It doesn't seem to pull data from masters? At least, not with the item lists. It'll reference formID's taken from the file it's part of, but not any of the master. Can't set a reftype due to it referencing multiple types of data, don't know if that'd fix it.

» Sat May 26, 2012 11:18 pm

BODT in both ARMO and ARMA pertain to body slot information. I don't know details...

» Sat May 26, 2012 7:34 pm

Uploaded a new version due to a bug in handling of lstring data types which would cause a lot of errors so updating is highly recommended. Also added a requested multiple subrecord copy/paste.

Obviously there is much room for improvement a big part of these latest updates was redoing the recordstructure.xml file to match skyrim more closely. In doing that I lost a lot of the work that was in the old file which is why I still deliver an old copy so you can reference it if you feel like updating your copy but there was so much that didn't match I thought it might be better to start over for a lot of the records.

I programmatically tried to detect data types to better anolyze what it is likely to be present but lstrings are frequently showing up in places it really shouldn't because multple records were able to find their index in string files so I guessed wrong. I'll be making another pass through the file but it is very time consuming with so many records. I'm aware of uesp.net and reference it when possible still takes quite a bit of time.

For those with the aptitude and willingness, feel free to post XML code snippets or PM me with bug fixes to the XML and I will try to incorporate those regularly.

Edit: Regarding the resizing problem. I'm surprised I didn't see that before. I was using variable binding to save the positions and the code seems to be reloading those due to selection changes. I will remove some of that so it wont do it all of the time.

Edit 2: trira thanks for that specific link. Searches kept landing me in TES4 which aren't nearly as useful so I wasn't using the site as much.

» Sun May 27, 2012 3:45 am

Any idea what is the flag for races to allow them to talk? Or is it a setting in the _NPC record?10x

» Sun May 27, 2012 5:20 am

You have no installation instructions. I can't get it to work because of this.

» Sun May 27, 2012 3:38 am

You have no installation instructions. I can't get it to work because of this.

Extract the archive and run the .exe file. No installation needed.

Edit: Something's wrong with newest version. Won't startup properly. The window seems to open outside of my screen haha

Edit2: Edit the conf/settings.xml file.

This line:

Should be:

Least, that worked for me.

» Sun May 27, 2012 7:39 am

Wait, thought i had dot net 3.5 but i don't. The requirements are absent from the readme.

According to the interwebz my Win 7 64bit DOES have dot net 3.5 but i can't get the program to run.

» Sat May 26, 2012 11:53 pm

Wait, thought i had dot net 3.5 but i don't. The requirements are absent from the readme.

According to the interwebz my Win 7 64bit DOES have dot net 3.5 but i can't get the program to run.

Look at my post. Edited it again.

» Sun May 27, 2012 12:35 am

Yep, that was it. i was just about to come on and say that it WAS loading up but was appearing off screen. Thanks.

edit: Damn. i was hoping this program would actually display cell coordinates for references. I miss TES4Gecko.

» Sat May 26, 2012 10:52 pm

Sorry about the conf file. It was using it on my second monitor and it saved its settings in the delivered file. That was left over code from original software.

I've got a lot of updates to the xml coming soon based on a synchronization with uesp.net wiki. I will make sure that this fix is present.

Edit: What exactly do you want to see regarding cell references? It might not be too difficult to add.

» Sat May 26, 2012 4:05 pm

It might not be possible yet. I don't have an intimate knowledge of the inner workings of TES4Gecko and it's entirely possible that the Cell coordinates listed in that program are converted from formids. Anyway that is what I'm looking for, cell coordinates.

Also just to be clear, I'm not disrespecting your effort in the least though it may seem that way on the surface. I'm very grateful for anything provided to me free of charge. Thanks.

» Sun May 27, 2012 6:54 am

cell coordinates, as in the cell inside a worldspace? Or coordinates for objects inside a cell? If the latter, that's contained in the OBND (object bounds) for references.. Or something like that.

» Sat May 26, 2012 7:25 pm

I've uploaded a new build hopefully with fewer crashes and other issues though I will not deny it needs more testing time. I've gone though just about every record and synced where possible with uesp.net wiki.

» Sat May 26, 2012 10:14 pm

cell coordinates, as in the cell inside a worldspace? Or coordinates for objects inside a cell? If the latter, that's contained in the OBND (object bounds) for references.. Or something like that.

Cell coordinates as in 0,0. Only the cell records contain the cell coordinates in the info panel. I'd like to see them in the location records if possible. Specifically for the LCSR subset, especially LCRT (MapMarkerRefType).

» Sun May 27, 2012 3:35 am

I just realized that I totally skipped all of the things that I tend to remove when playing with armors and weapons. Next version will add NAVI, CELL, REFR, WRLD, LAND, PHZD, PGRE and any others I find along the way. Now the wiki indicates that LCRT is like a color or something. Is that wrong?

I tried TES4Gecko and could not figure out what you were looking referencing specifically so I'm a little blind about what your asking for. I sort of can guess but I'm really not certain.

» Sat May 26, 2012 9:21 pm

I just realized that I totally skipped all of the things that I tend to remove when playing with armors and weapons. Next version will add NAVI, CELL, REFR, WRLD, LAND, PHZD, PGRE and any others I find along the way. Now the wiki indicates that LCRT is like a color or something. Is that wrong?

I tried TES4Gecko and could not figure out what you were looking referencing specifically so I'm a little blind about what your asking for. I sort of can guess but I'm really not certain.

Forget about Gecko.

LCRT = color? I don't think so. LCRT I would imagine stands for "location reference type". The LCRT records are sub records of LCTN and the subset is called LCSR. I'm surprised your not familiar with them because your program includes details like (WEScene), (MapMarkerRefType) and (Boss) for the LCRT subrecords. What I'm asking for is the LCTN records to include the cell coordinate in the list of data.

Honestly though, I'm just being selfish with this request to trying and lessen my own workload and since your obviously not understanding me I think we can safely just forget about my request. Thanks for your time anyway and good luck with the development of this fine and useful tool.

Cheers.

» Sat May 26, 2012 7:32 pm

I'm not the original author so I'm pretty green about the internal workings of ESM and ESP files and their contents. The reverse engineering of the internals of the files is more due to other modders that have updated to the uesp the wiki or the previous games. timeslip was original author with others making changes apparently and really this my first foray into this part of modding of TES games (usually stuck with NIF models or user interface mods in previous games). This is just my fork of the original project which is in mothballs as far as I can tell.

I can handle what requests I accept and reject. Feel free to make requests and I'll try not to lead you on about doing them. If its useful to you it may be useful to me in the future or others. At this point I'm just trying to understand what you want displayed. If its too hard or "expensive" then I'll not do it. I think this sounds pretty easy once I get an clear idea what you want shown.

I suspect that (WEScene), (MapMarkerRefType) and (Boss) are the Editor IDs of the records. The lookup routine will match the id and show the name. It could also just search the subrecords and pull out some extra information as well like your grid reference. I think you are asking for the "Location" column that would be displayed in the CK under the Cell View for exterior cells but I think I'm just being dense.

I've been struggling with the REFR, CELL, ACHR records as they do not conform to standard format which the current code cannot handle well so it needs a few updates to allow more arbitrary formatting. Once I'm happy with where that stands I'll do a little more here on this request.

» Sun May 27, 2012 12:41 am

I seem to be having issues with plugins corresponding to the masters; any lookups only search the local plugin, not the master, too. My only criticism thus far. And the default de-selecting of CELL and WTHR and IMAD, jeez.

I've been editing those for a while now. <_<

(btw; that's not criticism, I understand the reasoning and it was only one of those things that struck me as humorous).

» Sun May 27, 2012 4:51 am

I just realized that I totally skipped all of the things that I tend to remove when playing with armors and weapons. Next version will add NAVI, CELL, REFR, WRLD, LAND, PHZD, PGRE and any others I find along the way.

I assume this is the reason why all terrain is set to 0 when saving a plugin? Literally all the land is completely flat, even if I just resave my plugin without modifying anything. The file size is also bloated significantly after saving, without having made any changes.

» Sat May 26, 2012 6:34 pm

File size results from tessnip saving without the compressed flags, which is unrelated to this.

» Sat May 26, 2012 10:49 pm

It goes from 240 MB (it's a huuuge worldspace imported into Skyrim with TesAnnwyn) to 800... And all the land becomes flat.

» Sat May 26, 2012 11:03 pm

@plutoman, I will double check the form lookup routine. I remember encountering a some code there that looked strange. I'll make sure I didn't make it more incorrect or fix if possible.

As far as I am aware it does not actually alter the contents of the file unless you open it. I've not actually tried resaving the master after opening it. It might be some of the IMAD items which have illegal characters in the names and that causing issues.

I've not actually touched any of the load or save routines other than to add the load bypass for specific named cells. I've got a new build I will be releasing soon after I go through some validation checks but it claims to load all of the missing cells.