Havok Command Line Tools 1.3
============================================

Links:
http://www.gamesas.com/topic/1336657-rel-havok-animation-converter-thread-2/


This tool enables converting Havok animation from the Skyrim-compatible binary format to the Havok XML format or to the Gamebryo KF animation format.

Currently the package only converts from native HKX files built with Havok 2010.2.0 SDK to the XML form of HKX or to the Gamebryo KF file format.

This is useful because it enables animators base animations to work with in their tool of choice.

Prerequisites:

• For Skyrim, you must unpack the BSA files using an appropriate tool. The files must be unpacked exactly as they are stored in the BSA with same folder structure.
  For me I will refer to this location as "%TEMP%\Skyrim - Animations"
  
• I assume you have the 2010.2 Havok Content Tools and SDK from Intel.
  You will need to provide infomation to get access to the tools but is free.
  http://software.intel.com/sites/havok/en/
  
  
• For 3ds max support get the niftools.org exporter for Skyrim
  
  • http://www.skyrimnexus.com/downloads/file.php?id=5622 [skyrimnexus.com]
    
    • This is my persional fork of the importer. You might still get the official one from niftools.org if one is available.

Quick Start:

• Unpack BSA Files to a Folder
  
  • First unpack the Skyrim - Animation.bsa and Skyrim - Meshes.bsa files to another location
    
  • I will use my simple unpack tool to demonstrate this but other BSA tools presumably will work just need to maintain the folder structure
    
  • Use a folder other than temp if you like but working folders tend to be different for everyone
    

  • bsaunpak "c:\Steam\SteamApps\common\skyrim\Data\Skyrim - Animations.bsa" "%TEMP%\skya"			bsaunpak "c:\Steam\SteamApps\common\skyrim\Data\Skyrim - Meshes.bsa" "%TEMP%\skym"

• Convert Packed HKX to XML HKX files
  
  • If you do not specify the output folder it will assume that it should write to the same folder and append "-out" the filename.
    
  • The XML HKX files can be loaded into Havok tools like the Havok Preview Tool
    

  • hkxcmd convert "%TEMP%\Skyrim - Animations"

• Convert Packed HKX to XML HKX files and place in another folder
  
  • The following will convert all HKX files in the "%TEMP%\Skyrim - Animations" folder and subfolders and place them in the "%TEMP%\Skyrim - Animations - Output" folder.
    

  • hkxcmd convert "%TEMP%\Skyrim - Animations" "%TEMP%\Skyrim - Animations - Output"

• Preview Animation in Havok Preview Tool
  
  • Load the Preview Tool: Start | Havok | Havok PcXs Content Tools 2010.2.0 | Havok Preview Tool
    
  • Convert Skeleton.hkx using step 1 or 2 above
    
  • File | Open... | Browse for converted skeleton
    Assuming you used Tutorial 2 that would be the following for the chicken
    "%TEMP%\Skyrim - Animations - Output\meshes\actors\ambient\chicken\character assets\skeleton.hkx"
    If you get a "Wrong platform for packfile" error when loading then you selected the original skeleton not the converted one.
    
  • File | Add... | Browse for an animation for that skeleton
    Assuming you used Tutorial 2 that would be the following for the chicken
    "%TEMP%\Skyrim - Animations - Output\meshes\actors\ambient\chicken\animation\getup.hkx"
• Convert HKX file to Gamebryo KF files
  
  • The following will just dump KF files into same folder as the HKX file. Note: Use the original files not the XML files since those don't load currently.
    

  • hkxcmd exportkf "%TEMP%\Skyrim - Animations"

• Convert HKX file to Gamebryo KF files into a different folder
  
  • The following will convert all HKX files in the "%TEMP%\Skyrim - Animations" folder and subfolders and place them in the "%TEMP%\Skyrim - Animations - Output" folder.
    

  • hkxcmd exportkf "%TEMP%\Skyrim - Animations" "%TEMP%\Skyrim - Animations - Output"

• Import Skyrim Animation into 3ds Max
  
  • First import the creature into max using niftools max importer
    - Select Import | "Netimmerse/Gamebryo (*.nif, *.kf)" | Pick the file to import
    - Continuing the chicken theme I used the following:
    - "%TEMP%\Skyrim - Meshes\meshes\actors\ambient\chicken\character assets\chicken.nif"
    
  • If you have problems with meshes try importing the skeleton explicitly before the mesh
    - Not all creatures use the name skeleton.nif so you might have to manually adjust path
    
  • Assuming this succeeded now import the animation
    - Select Import | "Netimmerse/Gamebryo (*.nif, *.kf)" | Pick the animation
    - Find an animation you which to import such as the getup animation
    - Assuming you used Tutorial 5 that would be the following for the chicken:
    - "%TEMP%\Skyrim - Animations - Output\meshes\actors\ambient\chicken\animation\getup.kf"
• Export Skyrim Animation from 3ds Max in KF format
  
  • Select the root node (usually "NPC Root [Root]")
    
  • Open Graph Editors | Track View - Dope Sheet
    
  • Add a Note Track to the Root Node
    
  • Note that the KF generated by the exportkf process will now add start and end note tracks
    
  • Add a note at time 0 with text "start"
    
  • Add a note on the final frame of animation with text "end"
    
  • Select Export | "Netimmerse/Gamebryo (*.nif, *.kf)" | Specify an file name like "export.kf"
    
  • Select "Single KF w/o NIF" under Animation in the "Export Nif" dialog
    
  • Select "Export"
    
  • I usually load in Nifskope after this just to make sure its valid enough
• Convert KF format to HKX Format
  
  • The following will convert a KF file using the base skeleton for its bone layout and bind pose
    
  • By default it exports as a Win32 Packed Binary HKX file but you can change that from the command line to export as XML adding -v:XML
    

    hkxcmd convertkf "\path\to\skeleton.hkx" "\path\to\export.kf" "\path\to\export.hkx"

  Troubleshooting:

• A lot of files show that they failed to load when converting.
  - Ignore any errors about files it cannot convert as not every file can be read with the sdk.
  - This is because the SDK only includes Physics and Animation support. Bethesda also uses Havok Behaviors which are not supported by the SDK and those file cannot be read currently.
  
• I keep getting the following message "Wrong platform for packfile" when using the Preview tool
  - You are loading the original animations from Bethesda which are not directly compatible with the preview tool. Follow the tutorial parts 1 and 2 and export the XML HKX files and use those.
  
• The animation is corrupt after importing into 3ds Max
  - Try Importing the Skeleton.NIF file explicitly before importing the mesh.
  - Some animations just don't import correctly probably because of bone mapping issues
  
• When I try to load the converted HKX file into Havok Preview Tool fails.
  - The Havok Preview Tool breaks with large files like the dragon.hkx files.
  - You might be able to convert the file to a standard packed file with the -f switch on Convert command.

Known Issues:

• Some reports than human animation does not work. Embarrassingly I didn't try humans only creatures.
  
• I could not figure out how bones are mapped because I'm not sure the information is present. I've just assumed that there is a 1-to-1 skeleton to animation track but that is obviously not always a correct assumption
  
• Havok supports non-uniform scale in animation however Gamebryo does not. This means any animation exported to the Gamebryo format may lose information when this is present.
  
• KF was choosen because the niftools.org 3ds max importer largely works and this was the easiest way to do this porting. It may be worth while converting to FBX format instead but that is much more work but would presumably have broader support.
  
• There is no handling for visbility or camera tracks which seem to be used

Acknowledgements:

• Niftools team for niflib and the 3ds Max Importer
  
• Havok and Intel for the Havok SDK and perpetual license for 3rd party tools

Change Log:

• 1.3
  
  • Change ExportKF to add "start" and "end" note tracks on root node for later export
    
  • Added ConvertKF which takes a Gamebryo KF file and produces an HKX file
    
  • Added DumpText which is a text file dump of bones in a skeleton may be useful in the Havok Export Pipeline when specifying the skeleton.
• 1.2
  
  • Redid the directory handling for exportkf
• 1.1
  
  • Fixed crash when no output folder was specified when using the exportkf command

Downloads:
http://www.skyrimnexus.com/downloads/file.php?id=1797 [skyrimnexus.com]
http://www.mediafire.com/?gvy4yxt8d7tnbsc [mediafire.com]

Optional Downloads:
http://www.mediafire.com/?54w2gnjvovlddv9 [mediafire.com]

Hello,

This is very cool, I'll have to try it out this morning. I've only managed to convert the .hkx files to .xml with the htcStandAloneFilterManager.exe that came with the Havok Content Tools. This will be a nice way to get animations into 3dsmax. Many thanks.

Best regards,
RX31

Nice one!

I have been able to load them just fine. Just check out what version hct was used to create them... you need to use version 2010.2 of the havok tools.

I'm pretty sure I have the correct version of the Havok Content tools (2010.2.0) but cannot load any of the files without converting them first. And never could quite get the gist of that filter tool. But I just ran the converter on everything and dont have a problem with my converted files. The only interesting thing to me now would be if its possible to load the files that have Behaviors attached and convert those to XML.

Can anyone confirm that this is anything other then Malware ? it consists of just a couple txt's and an exe I am weary of touching from someone who's name I have zero memory of ever being here.

Also an actual example file of how to use the program would help all you give are a bunch of cmd lines that I ran from behind a quarantine that did absolutely nothing...

A 10 second review of the file makes it look like it probably does what it says it does. Also, the source is at https://github.com/figment/hkxcmd.

I'm not going to say its perfect but I did give about 6 examples in the readme and in the post above. Unfortuately I cant also unpack the files as well to complete the tutorial and I cant be bothered to write a user interface for something that 5 people will probably use once to unpack the files.

Each of the > lines are commands that can be run from Windows command prompt. Here is the http://www.virustotal.com/file-scan/report.html?id=01270e308b4446bb9bbbad5d2dfac49633276930ad279c5a683664aeed20c554-1322441878 report if you dont really trust me. Its 10 MB because its statically linked with Havok and Niftools which are each 5 MB monstrosities.

The one you are most interested in is #5:
path\to\hkxcmd.exe exportkf "path\to\Skyrim - Animations" "path\to\output"

Where it sort of expects you to unpack the complete "Skyrim - Animations.bsa" file to a folder. For it to find the skeleton and the animation files it makes assumptions about where they are.

If you prefer to do things manually then you can do something like:
path\to\hkxcmd.exe exportkf "path\to\Skyrim - Animations\meshes\actors\dragon\character assets\skeleton.hkx" "path\to\Skyrim - Animations\meshes\actors\dragon\animations\ground_exhalefirebreath.hkx" "output.kf"

If you need other help the you can run "hkxcmd.exe Help ExportKF" for some assistance but it really wont tell you much.

It just doesn't do very much but the one thing which is converts files and that is not very visual. Here are some youtube links of my some animations playing in my max environment.
http://www.youtube.com/watch?v=YpOYpI5hrwY http://www.youtube.com/watch?v=ahA50HkuDU4

Edit: Actually the easiest way to run it is to copy the exe into the "Skyrim - Animations" folder. Open a command prompt. CD to the animations folder and run "hkxcmd exportkf . ." where the . is important. That will put all of the KF files in your extracted folders.

Also looks like I introduced a bug where it expects you to provide the output directory instead of guessing it. Will fix for next release.

Ok, I removed it from quarantine and it works~ but it seems to have bugs with ExportKF, Convert Works but the exact same paths with ExportKF kick back all kinds of errors like "Skeleton file not found" ?!? both of them are there or it kick back "no files found" or just just crashes.

For instance

Works
hkxcmd convert "SK" "SK-Out"

Does not work
hkxcmd exportkf "SK" "SK-Out"

also its completely unable to read custom.hkx files they are structured similar but the memory resource container is above the animation container.

Fair enough. I agree its bugger than I'd like and needs a little more baking time. Made a number of last minute changes to be cleaver and its causing problems with the directory detection.

I'll see what I can do about supporting files where the hkAnimationContainer is not the root object. Also human animations do seem to have problems in my testing. Probably related to the bone index issue referenced in the Known Issues.

Edit: I see what the problem was with the exportKF. I forgot to call the function create missing directories. My test runs were usually convert first then export into same directory. Bummer

Edit2: Another round of updates. Feel free to wait it out but should be a little better now. And apologies in advance for any bugs. Have not tackled non standard .hkx files yet.

Also changed it so that it looks for skeleton.hkx by default only uses that file and finds animations relative to that file (..\animations) rather than how it did before which was looking for *project.hkx files and looking relative to that. The human character animation seem to work properly now on my exports.

Its Behavior Bone LOD its screwing up the Bone Indices @Export and since we have no access to Behavior we are basically F***ed until a 3rd party tool or NifTools gets updated to deal with it.

I don't too much about what going on in havok despite this tool but could this be handled via explicitly specifying m_transformTrackToBoneIndices in the hkaAnimationBinding in the exported hkx file? Bethesda never specifies it so I've assumed that track=bone but this presumably could alter that behavior. If Bethesda/Havok uses it then it could help. But agreed this requires 3rd party helper tool I think.

Edit: It should be possible to take an existing animation file and manipulate it directly without the max import/export step and specify garbage to see messes up the animation to if bethesda uses it or not.

Do we know if the game will take the XML HKX files for import instead of the binary packed when overriding?

Hey, nice work, thanks Hologram! (unfortunately, sticking this in my pile of things to look at later)

I found a free version of the Havok Behavior Tool its ancient version 6.6.0 but may prove somewhat useful for anolysation.

http://forum.cgpersia.com/f13/havok-tools-sdk-3533/

Havok has a page for being able to DL the official Behavior Tool but the links leads to dead ends :/.

HBT 6.6 was what was on the Havok DL page back around a year ago when I DLed. Not sure how old it is really, but it was their latest DL not more than a year ago. I had made a project folder for all the SKyrim characters a while back, and couldn't get them to load.

Its not going to be directly useful for Skyrim or any game really -> but it does come with a few tutorial and sample files that will allow someone to create simple behavior.hkx files and take them apart and see if we can cobble together a very basic HBT States and Transitions Editor that would at least allow some basic editing/creation of HBT Behaviors for new creatures.

That is a pretty useful link. The app is .NET and the layout of the Havok classes are in the Havok assembly and seem to match fairly closely to the SDK headers. Would be useful in potentially creating a compatibility layer for the behaviors bethesda uses. Even better is whether the XML hkx files could be converted to the hk661 format and then loaded into that HBT. If not directly could an approach like the one I took be used to bridge the gap? (ie hkx v2010.2.0 -> XML -> hkx v661) Not sure I have time to immediately try that but might be feasible.

That maybe viable but the problem is just how much data would be transferable going the opposite direction considering our point here is to get assets in-game not into HBT although we would need a viable path both directions.

Definitely agree but I'm looking at different possibilities for the future. Something that is always asked for is custom animations for different new actions. Not sure that HBT is right tool but seems like that would help with adding transitions for such a modification. I still think that more investigation into use of the transformTrackToBoneIndices array is a potentially good lead on pulling in animation in the first place with different indices. The other is to figure out if Havok uses a custom way of specifying bone indices for export via User Properties or such. I've seen that technique in games using Granny as I recall.

2 Things I am absolutely sure about !

1) The CK has some way of defining animatable nodes per actor, because if you take the player skeleton, animations, behaviors and character profiles and put them into the chicken folders <- ONLY the nodes shared by both the Player and the Chicken animate everything else is completely ignored even though every asset in play was designed to work together.

2) there is absolutely some unknown bit in the animations themselves that helps with mapping indices to nodes and every option in the HCT Plugin is not including this in the output files but I am missing some important Beth things like their RigProfile and any integration into Behavior itself that HCT does not include.

After some additional digging around the SDK I found Tools\BatchProcess\AssetCc\AssetCc2.exe which is a standalone converter tool similar to my convert option but obviously better in many regards. It has native support for many behavior classes so it can convert files that I cannot read at this time. Project.hkx and in the characters folders can be converted using it however behavior*.hkx still crash when using it so that implies customization on Bethesda's part for these files.

If you think about it BS specific customization would be required as they had to finagle hkx animations to function within whatever version of Actor BoneLOD they are using plus to get things to properly map from an animation.hkx to the skeleton.nif.

Just a heads up on what I've been working on. I have a KF to HKX/XML converter mostly working. The main problem I have is that you have to add note tracks in max to make the animation be willing to export with niftools exporter. Rather an inconvenient step so maybe I need to add an option on export to just assume things if I continue to use this approach. But the animation seems to be correctly exported and plays well with original skeleton in limited testing.

I've also been trying to remove the KF file from the loop and just do the same operation with freshly exported files but that is not going as well for some reason I've yet to determine why since I do even less manipulation. This would be good to get working because its then just a matter of setting up the havok export pipeline with the custom converter as the last step of that pipeline.

I've noticed that there are some issues with the KF generator where TimeScale is not quite right. I used duration/frames to get incremental frame length (0.03333 s/fr in most animation where it actually seems to expect 0.0325 s/fr) so the KF files need to be reworked so that they can be exported cleanly again.

I get a lot of scale/skew warnings when exporting animation with Havok but Max plays animation back ok so I think I need to verify that I'm putting normalized quats into the KF used to import. I wouldn't expect to have to do this as I'm getting the quaternions from Havok in the first place and it likes normalized quats.

Just posted a new version 1.3 with the KF to HKX conversion enabled. Quick Start bullets 7 and 8 added in the first post. At this point, I've successfully imported the human skeleton and one of its animation into max and then exported from max and converted to an hkx file that was successful used by Skyrim. Below is brief breakdown of the process I followed with some more concreteness than before:

• Exported "Skyrim - Animations.BSA" files to disk to "d:\temp\skya" and "Skyrim - Meshes.BSA" to "d:\temp\skym"
  
• Placed a copy of hkxcmd.exe into d:\temp
  
• Opened "Command Prompt" with working folder at D:\Temp
  
• Run in Command Prompt:

  hkxcmd exportkf "d:\temp\skya" "d:\temp\skykf"

• Open 3ds Max 2010 x64 (Version should not matter)
  
• Import "d:\temp\skym\meshes\actors\character\character assets\skeleton.nif" using 3.7.1 nifools.org max importer
  
  • Used defaults
• Import "d:\temp\skykf\meshes\actors\character\animations\1hm_attackpower.kf"
  
  • Used defaults
    
  • Note that the 1.3 version of exportkf will add "start" and "end" note tracks to the "NPC Root [Root]" node
• At this point we would normally manipulate the file if we wanted to make changes.
  
• In preparation for export we would normally have to select the "NPC Root [Root]" node and edit its Note Tracks to added "start" and "end" notes to mark the animation range (and root node) but the importer pulled them in for us
  
• Export KF file to "D:\temp\export.kf" using Export NIF dialog
  
  • Select "Single KF w/o NIF" in Animation section
• Preview in Nifskope
  • Open "D:\temp\export.kf" to verify it loads without errors and seems to have content
    
  • Normally I would load "d:\temp\skym\meshes\actors\character\character assets\skeleton.nif" and then attach the animation to test but that does not work with these files in 3.7.1 by default.
• (Optional) Preview the code in Havok Preview Tool
  
  • Run in Command Prompt:

    hkxcmd convert "D:\temp\skykf\meshes\actors\character\character assets\skeleton.hkx" "d:\temp\base_skel.hkx"hkxcmd convertkf -v:XML "D:\temp\skykf\meshes\actors\character\character assets\skeleton.hkx" "d:\temp\export.kf" "d:\temp\export.hkx"

  • Exporting as xml file is optional and used if you want to inspect the content of the file
    
  • To preview the animation in the Havok Preview Tool. I have to convert the base skeleton to and animation to xml
    
  • File | Open "d:\temp\base_skel.hkx"
  • File | Add "d:\temp\export.hkx"
• Run in Command Prompt:

  hkxcmd convertkf "D:\temp\skya\meshes\actors\character\character assets\skeleton.hkx" "d:\temp\export.kf" "C:\steam\steamapps\common\skyrim\data\meshes\actors\character\animations\1hm_attackpower.hkx"

  • By default I'm using the default Spline Compression settings. In the future I will try to provide a means to specify some compression settings
• Open Skyrim. Equip a dagger and power attack the air. Looks basically identical to the original animation which is what I want at the moment

Problems I had:

• Weapon name was case sensitive and had crappy rotations by default. This was fixed but I could see Shields or Quivers or something messing up in similar ways that need to be checked
  
• Camera3 was being put into the animations and totally screwed up the 3rd person perspective when in game so I put in a kludge to deal with it by excluding siblings to root node

Still having issues with ExportKF not working at all.

hkxcmd convert .. <- works although xml files are useless to me.
hkxcmd exportkf .. <- kicks back no files as an error, even if I specify exact files and folders to input/output it still fails.