[WIPz] Skyrim Script Extender (SKSE) : V - Skyrim

» Wed May 16, 2012 3:35 pm

While it's great that you're going to be working on this, I'm going to be more interested in finding out what the new Creation Kit's supposedly brand-new scripting language can do. I'm hopeful they made some big improvements. I think they may have made some really big, nice improvements in connecting actors (and other items) to animation nodes; I think that's how they handle weapon racks and possibly sitting in a moving cart and such. If so, that's awesome.

Anyways. Gotta see what the actual scripting language is like before jumping on a script extender.

I actually agree. I can't wait to see what they have done. Do we get FormLists like we had in Fallout? String variables? Loops? Can we define our own functions? Is it still a c-style function calling script style, or is it object-oriented? Maybe they just have a python-like language and there is very little for a script extender to do? If that is the case we can call it a win all the way around.

However I imagine there will still be some bits and pieces of the world that haven't been fully exposed yet. We can expose those. Can you script the UI and build new interfaces dynamically through code? If not, we can work on that. What about allowing custom spells to be created?

Chances are good that there are areas that can use some extension. That is what we are here for.

» Wed May 16, 2012 12:32 pm

Yo! behippo, scruggsy, ianpatt et al - delighted to see you're all still into this. Every mod I've done has required OBSE/FOSE, so I'm really looking forward to seeing what you can do for Skyrim.
Mind you, I'm firstly looking forward to seeing what the Creation Kit can do out of the box.
Maybe Beth will take on board a lot of what you guys did for earlier games, and include the more important of your *SE scripting options from the get-go.

Anyhoo - I guess we're all salivating in anticipation. Happy modding, y'all, and really sincere thanks to the script-extender crew for your work over the years.

(Edit: stupid typos)

» Wed May 16, 2012 6:32 am

Just in case the SKSE team want to check that the exe file is the same across regions. I live in the UK. The MD5 of TESV.EXE is 540e684cb63b0218ec865c7791649734 and the memory offset of the command table is data:0x01426170 with the command "Gamemode".

» Wed May 16, 2012 3:12 pm

I actually agree. I can't wait to see what they have done. Do we get FormLists like we had in Fallout? String variables? Loops? Can we define our own functions? Is it still a c-style function calling script style, or is it object-oriented? Maybe they just have a python-like language and there is very little for a script extender to do? If that is the case we can call it a win all the way around.

If I recall correctly, Lord Viper alluded that the new language would take a few cues from UE3's UnrealScript/Kismet :ahhh:

*gibbers madly*

Well, here's hoping it's similar enough to OB(SE)Script - Less work for me when I start porting CSE to the CK :spotted owl:

» Wed May 16, 2012 2:54 pm

Looks like the UK runtime is the same as the US runtime. Most likely the German no-gore version is different again, though.

Time to start turning this in to the "Skyrim Internal Technical Research Discussion" thread. This time around, it looks like we have two totally different scripting engines in the game. One is the classic system used in the previous games, it looks almost exactly the same from a very quick lookthrough. Command dump coming shortlyish.

The new one is much more interesting and appears to be called Papyrus or SkyrimVM. We'll probably need to wait for the editor to be released to get any idea about the syntax as the original source code for scripts doesn't appear to be in the .esm (unless they're compressed). However, here's a quick feature rundown from what I can tell. Please take things with a grain of salt for now, this is just a first look.

core variable types are float, int, string, and bool
arrays of the above are supported natively
scripts are interpreted as a list of simpler yet much more powerful set of opcodes
if the conditional branch opcodes support negative offsets then loops are trivial to implement (and probably have been)
the new system is object-oriented - all scripts are class instances (UnrealScript influence here)
compiled scripts are stored as separate files, unpack Misc.bsa and look in the scripts folder
classes support inheritance, and there appear to be vm-world classes matching the internal form types

So overall this is looking very, very good. I need to write something to disassemble the compiled scripts and take a look at the VM runtime, will know more then.

It looks like a lot of the basic functionality of OBSE (etc) is present, mainly by it being closer to an "actual" programming language than the previous system, however it looks like there's still a lot that we can do to expand on its functionality. Massive thanks to SmkViper on this, it's looking really nice so far (and thanks for the error messages when the script loader doesn't like something).

» Wed May 16, 2012 1:28 am

Looks nice indeed Ian! I'm anxious to take a look myself - but my game has not yet arrived.

Can't wait to get to work.

» Wed May 16, 2012 9:19 am

http://skse.silverlock.org/vanilla_commands.html.

» Wed May 16, 2012 5:09 pm

Looks like the UK runtime is the same as the US runtime. Most likely the German no-gore version is different again, though.

Time to start turning this in to the "Skyrim Internal Technical Research Discussion" thread. This time around, it looks like we have two totally different scripting engines in the game. One is the classic system used in the previous games, it looks almost exactly the same from a very quick lookthrough. Command dump coming shortlyish.

The new one is much more interesting and appears to be called Papyrus or SkyrimVM. We'll probably need to wait for the editor to be released to get any idea about the syntax as the original source code for scripts doesn't appear to be in the .esm (unless they're compressed). However, here's a quick feature rundown from what I can tell. Please take things with a grain of salt for now, this is just a first look.

core variable types are float, int, string, and bool
arrays of the above are supported natively
scripts are interpreted as a list of simpler yet much more powerful set of opcodes
if the conditional branch opcodes support negative offsets then loops are trivial to implement (and probably have been)
the new system is object-oriented - all scripts are class instances (UnrealScript influence here)
compiled scripts are stored as separate files, unpack Misc.bsa and look in the scripts folder
classes support inheritance, and there appear to be vm-world classes matching the internal form types

So overall this is looking very, very good. I need to write something to disassemble the compiled scripts and take a look at the VM runtime, will know more then.

It looks like a lot of the basic functionality of OBSE (etc) is present, mainly by it being closer to an "actual" programming language than the previous system, however it looks like there's still a lot that we can do to expand on its functionality. Massive thanks to SmkViper on this, it's looking really nice so far (and thanks for the error messages when the script loader doesn't like something).

Yes! This sounds absolutely fabulous. Thanks for the early tidbits.

» Wed May 16, 2012 6:16 am

Looks like the scripting language won't take long to get used to for modders, considering most of the console commands are identical to the Oblivion/FO3 ones. TCL, TGM, TM, TFC, TAI, show, timescale, gamehour, getav, setav, etc.

» Wed May 16, 2012 1:12 am

Super news. I can't wait to see what can be done with it.

Although it is slightly sad that I'm more excited about the mods than the game. And that's before I've even played it.

» Wed May 16, 2012 7:06 am

I've played it, and I already know I want something like More Hotkeys for creating equipment sets, so I can quickly switch between an armor outfit and clothing outfit with 2 keypresses.

» Wed May 16, 2012 6:24 am

core variable types are float, int, string, and bool
the new system is object-oriented - all scripts are class instances (UnrealScript influence here)

Epic woot!

» Wed May 16, 2012 11:38 am

This is great news! From what I've seen the game eill need some heavy, advanced modding, so I am glad that you guys are back!

» Wed May 16, 2012 3:11 pm

Super news. I can't wait to see what can be done with it.

Although it is slightly sad that I'm more excited about the mods than the game. And that's before I've even played it.

Somewhat like wise-- I spent more time modding Obliviont han playing it, I think

Good luck with this, y'all-- I suspect it will be very useful to some of my planned Mods.

You probably already intend it (I have also not read every post, just a skim)-- but can I please request an Oblivion-style 'IsKeyPressed3' function (I think that is the one for 'As long as it is pressed, regardless of other keys)?
I do not know if it was my system or what, but on NV I found that 'IsKeyPressed' almost never registed key presses... Fortunately it did not matter too much, as my Mods were personal use back then.

» Wed May 16, 2012 2:54 pm

Looks like the UK runtime is the same as the US runtime. Most likely the German no-gore version is different again, though.

Time to start turning this in to the "Skyrim Internal Technical Research Discussion" thread. This time around, it looks like we have two totally different scripting engines in the game. One is the classic system used in the previous games, it looks almost exactly the same from a very quick lookthrough. Command dump coming shortlyish.

The new one is much more interesting and appears to be called Papyrus or SkyrimVM. We'll probably need to wait for the editor to be released to get any idea about the syntax as the original source code for scripts doesn't appear to be in the .esm (unless they're compressed). However, here's a quick feature rundown from what I can tell. Please take things with a grain of salt for now, this is just a first look.

core variable types are float, int, string, and bool
arrays of the above are supported natively
scripts are interpreted as a list of simpler yet much more powerful set of opcodes
if the conditional branch opcodes support negative offsets then loops are trivial to implement (and probably have been)
the new system is object-oriented - all scripts are class instances (UnrealScript influence here)
compiled scripts are stored as separate files, unpack Misc.bsa and look in the scripts folder
classes support inheritance, and there appear to be vm-world classes matching the internal form types

So overall this is looking very, very good. I need to write something to disassemble the compiled scripts and take a look at the VM runtime, will know more then.

It looks like a lot of the basic functionality of OBSE (etc) is present, mainly by it being closer to an "actual" programming language than the previous system, however it looks like there's still a lot that we can do to expand on its functionality. Massive thanks to SmkViper on this, it's looking really nice so far (and thanks for the error messages when the script loader doesn't like something).

Woah! Wow, Yay. Great news. Thanks!

Sorry about the Emo outburst, I could not help myself.

» Wed May 16, 2012 6:58 am

Looks like the UK runtime is the same as the US runtime. Most likely the German no-gore version is different again, though.

Time to start turning this in to the "Skyrim Internal Technical Research Discussion" thread. This time around, it looks like we have two totally different scripting engines in the game. One is the classic system used in the previous games, it looks almost exactly the same from a very quick lookthrough. Command dump coming shortlyish.

The new one is much more interesting and appears to be called Papyrus or SkyrimVM. We'll probably need to wait for the editor to be released to get any idea about the syntax as the original source code for scripts doesn't appear to be in the .esm (unless they're compressed). However, here's a quick feature rundown from what I can tell. Please take things with a grain of salt for now, this is just a first look.

core variable types are float, int, string, and bool
arrays of the above are supported natively
scripts are interpreted as a list of simpler yet much more powerful set of opcodes
if the conditional branch opcodes support negative offsets then loops are trivial to implement (and probably have been)
the new system is object-oriented - all scripts are class instances (UnrealScript influence here)
compiled scripts are stored as separate files, unpack Misc.bsa and look in the scripts folder
classes support inheritance, and there appear to be vm-world classes matching the internal form types

So overall this is looking very, very good. I need to write something to disassemble the compiled scripts and take a look at the VM runtime, will know more then.

It looks like a lot of the basic functionality of OBSE (etc) is present, mainly by it being closer to an "actual" programming language than the previous system, however it looks like there's still a lot that we can do to expand on its functionality. Massive thanks to SmkViper on this, it's looking really nice so far (and thanks for the error messages when the script loader doesn't like something).

This is insanely exciting.

» Wed May 16, 2012 3:10 am

Looks like the UK runtime is the same as the US runtime. Most likely the German no-gore version is different again, though.

Time to start turning this in to the "Skyrim Internal Technical Research Discussion" thread. This time around, it looks like we have two totally different scripting engines in the game. One is the classic system used in the previous games, it looks almost exactly the same from a very quick lookthrough. Command dump coming shortlyish.

The new one is much more interesting and appears to be called Papyrus or SkyrimVM. We'll probably need to wait for the editor to be released to get any idea about the syntax as the original source code for scripts doesn't appear to be in the .esm (unless they're compressed). However, here's a quick feature rundown from what I can tell. Please take things with a grain of salt for now, this is just a first look.
.....
It looks like a lot of the basic functionality of OBSE (etc) is present, mainly by it being closer to an "actual" programming language than the previous system, however it looks like there's still a lot that we can do to expand on its functionality. Massive thanks to SmkViper on this, it's looking really nice so far (and thanks for the error messages when the script loader doesn't like something).

Wow, what a suprise. An actual real language this time. Good thing I took a crash course on object-oriented languages a while back.

Although I'm a bit confused, why would there be two different scripting systems?

» Wed May 16, 2012 6:00 am

Looks like the scripting language won't take long to get used to for modders, considering most of the console commands are identical to the Oblivion/FO3 ones. TCL, TGM, TM, TFC, TAI, show, timescale, gamehour, getav, setav, etc.

The 'old-style' scripting language is apparently almost identical. The new 'Papyrus' scripting language we know almost nothing about yet, except that it's awesomely more powerful than the old one. It's also going to be a bit different to script for, since it works so differently, but aside from being object-oriented, we don't know *how* different.

» Wed May 16, 2012 3:31 pm

Wow, what a suprise. An actual real language this time. Good thing I took a crash course on object-oriented languages a while back.

Although I'm a bit confused, why would there be two different scripting systems?

Perhaps the new system doesn't work for the console code? Perhaps the old system was too ingrained in the code and there was no need to scrub the code and take everything out? The old system worked for years and was pretty well strewn across the code base, especially for dumping info to the console for state changes and the like. Dunno - maybe Viper could say? Or perhaps it will be obvious as we get a chance to play with things some more.

» Wed May 16, 2012 2:39 am

Not to mention quick and dirty code snippets needed for dialogue results/etc. What I'm hoping is that you can reference Papyrus classes in old-style scripts. In any case, I understand why they have two different scripting languages, especially since the new one seems much more complex and, hey, they already had the old one in there, no need to completely gut it.

» Wed May 16, 2012 6:54 am

Not to mention quick and dirty code snippets needed for dialogue results/etc. What I'm hoping is that you can reference Papyrus classes in old-style scripts. In any case, I understand why they have two different scripting languages, especially since the new one seems much more complex and, hey, they already had the old one in there, no need to completely gut it.

A quick look at the vanilla script command table seems to indicate that there won't be much interop b'ween the two - Only the console appears to use it. (They have actually done away with the TESScriptableForm base form component all together).

» Wed May 16, 2012 6:08 am

I am hoping for support for "in-game modding".

In other words, I want enough abstraction to represent meshes and textures, and I want to be able to invalidate existing instances and replace them with updated copies.

Thus, for example: a dirt&blood mod that relies on the player character being a unique instance (probably a unique race that just happens to work like a stock race for any purpose other than rendering), and a clean base texture for skin, which updates the in-use texture with dirt and gore and so on based on in-game events (swimming would be different from getting hit in battle).

Or, for example, a clothing mod which lets you combine non-wearable pieces to make a wearable suit.

Or, for example, a smithing mod which lets you arrange overlapping blade components to form a sword which is then converted to a .nif.

Or, maybe a hair brush...

Or maybe a character generation mod that lets us try on different armor and see ourselves under various lighting conditions. (Then plug in the moph data into a save file for start of game, and load that...)

Anyways, the possibilities are overly ambitious, and my examples get into other issues, but this is what I am hoping for.

» Wed May 16, 2012 2:59 pm

Partial Papyrus Script Function List (or so it appears):

Spoiler

"AddInventoryEventFilter", "ActiveMagicEffect","ActiveMagicEffect", "AddInventoryEventFilter","Dispel", "ActiveMagicEffect""GetBaseObject", "ActiveMagicEffect""GetCasterActor", "ActiveMagicEffect""GetTargetActor", "ActiveMagicEffect""RegisterForAnimationEvent", "ActiveMagicEffect""RegisterForLOS", "ActiveMagicEffect""RegisterForSingleLOSGain", "ActiveMagicEffect""RegisterForSingleLOSLost", "ActiveMagicEffect""RegisterForSingleUpdate", "ActiveMagicEffect""RegisterForSingleUpdateGameTime", "ActiveMagicEffect""RegisterForSleep", "ActiveMagicEffect""RegisterForTrackedStatsEvent", "ActiveMagicEffect""RegisterForUpdate", "ActiveMagicEffect","RegisterForUpdateGameTime", "ActiveMagicEffect","RemoveAllInventoryEventFilters", "ActiveMagicEffect","ActiveMagicEffect", "RemoveAllInventoryEventFilters","RemoveInventoryEventFilter", "ActiveMagicEffect","StartObjectProfiling", "ActiveMagicEffect""StopObjectProfiling", "ActiveMagicEffect","UnregisterForAnimationEvent", "ActiveMagicEffect","UnregisterForLOS", "ActiveMagicEffect""UnregisterForSleep", "ActiveMagicEffect""UnregisterForTrackedStatsEvent", "ActiveMagicEffect""UnregisterForUpdate", "ActiveMagicEffect""UnregisterForUpdateGameTime", "ActiveMagicEffect""AddPerk", "Actor""AddShout", "Actor""AddSpell", "Actor","AllowBleedoutDialogue", "Actor""AllowPCDialogue", "Actor""AttachAshPile", "Actor""ClearArrested", "Actor""ClearExtraArrows", "Actor","ClearKeepOffsetFromActor", "Actor","ClearLookAt", "Actor""DamageActorValue", "Actor""DispelAllSpells", "Actor""DispelSpell", "Actor""DoCombatSpellApply", "Actor""EnableAI", "Actor""EquipItem", "Actor""EquipShout", "Actor""EquipSpell", "Actor""EvaluatePackage", "Actor""ForceActorValue", "Actor""GetActorValue", "Actor""GetActorValuePercentage", "Actor""GetBaseActorValue", "Actor""GetBribeAmount", "Actor""GetCrimeFaction", "Actor""GetCombatState", "Actor""GetCombatTarget", "Actor", "GetCurrentPackage", "Actor""GetDialogueTarget", "Actor""GetEquippedShout", "Actor""GetEquippedSpell", "Actor","GetEquippedWeapon", "Actor""GetEquippedShield", "Actor""GetEquippedItemType", "Actor""GetFactionRank", "Actor","GetFactionReaction", "Actor""GetFlyingState", "Actor""GetGoldAmount", "Actor""GetHighestRelationshipRank", "Actor""GetKiller", "Actor""GetForcedLandingMarker", "Actor""GetLevel", "Actor","GetLeveledActorBase", "Actor""GetLightLevel", "Actor""GetLowestRelationshipRank", "Actor""GetNoBleedoutRecovery", "Actor""GetPlayerControls", "Actor""GetRace", "Actor""GetRelationshipRank", "Actor""GetSitState", "Actor""GetSleepState", "Actor""GetVoiceRecoveryTime", "Actor""HasAssociation", "Actor""HasFamilyRelationship", "Actor""HasLOS", "Actor""HasMagicEffect", "Actor""HasMagicEffectWithKeyword", "Actor""HasParentRelationship", "Actor""HasPerk", "Actor""HasSpell", "Actor""IsAlarmed", "Actor" IsAlerted", "Actor" "IsAllowedToFly", "Actor" "IsArrested", "Actor" "IsArrestingTarget", "Actor", "IsBleedingOut", "Actor", "IsBribed", "Actor" "IsChild", "Actor" "IsCommandedActor", "Actor" "IsDead", "Actor", "IsDetectedBy", "Actor" "IsDoingFavor", "Actor" "IsEquipped", "Actor", "IsEssential", "Actor", "IsFlying", "Actor", "IsGhost", "Actor", "IsGuard", "Actor", "IsHostileToActor", "Actor" "IsInCombat", "Actor" "IsInFaction", "Actor" "IsInKillMove", "Actor" "IsIntimidated", "Actor" "IsPlayersLastRiddenHorse", "Actor" "IsPlayerTeammate", "Actor", "IsRunning", "Actor", "IsSneaking", "Actor", "IsSprinting", "Actor", "IsTrespassing", "Actor", "IsUnconscious", "Actor", "IsWeaponDrawn", "Actor", "KeepOffsetFromActor", "Actor", "KillSilent", "Actor", "Kill", "Actor", "ModActorValue", "Actor", "ModFactionRank", "Actor", "MoveToPackageLocation", "Actor", "OpenInventory", "Actor", "PathToReference", "Actor", "PlayIdle", "Actor",  "PlayIdleWithTarget", "Actor", "PlaySubGraphAnimation", "Actor", "RemoveFromFaction", "Actor", "RemoveFromAllFactions", "Actor", "RemovePerk", "Actor", "RemoveShout", "Actor", "RemoveSpell", "Actor", "ResetHealthAndLimbs", "Actor", "Resurrect", "Actor", "RestoreActorValue", "Actor", "SendAssaultAlarm", "Actor",  "SendTrespassAlarm", "Actor", "SetActorValue", "Actor", "SetAlert", "Actor", "SetAllowFlying", "Actor", "SetAlpha", "Actor", "SetAttackActorOnSight", "Actor", "SetBribed", "Actor", "SetCrimeFaction", "Actor", "SetCriticalStage", "Actor", "SetDoingFavor", "Actor", "SetFactionRank", "Actor", "SetGhost", "Actor", "SetHeadTracking", "Actor", "SetForcedLandingMarker", "Actor", "SetIntimidated", "Actor", "SetLookAt", "Actor", "SetNoBleedoutRecovery", "Actor", "SetNotShowOnStealthMeter", "Actor", "SetOutfit", "Actor", "SetPlayerControls", "Actor", "SetPlayerResistingArrest", "Actor", "SetPlayerTeammate", "Actor", "SetRace", "Actor",  "SetRelationshipRank", "Actor", "SetRestrained", "Actor", "SetUnconscious", "Actor", "SetVehicle", "Actor",  "SetVoiceRecoveryTime", "Actor", "ShowBarterMenu", "Actor", "ShowGiftMenu", "Actor", "StartCannibal", "Actor", "StartCombat", "Actor", "StartVampireFeed", "Actor", "StopCombat", "Actor",  "StopCombatAlarm", "Actor", "TrapSoul", "Actor", "UnequipAll", "Actor", "UnequipItem", "Actor", "UnequipShout", "Actor", "UnequipSpell", "Actor", "UnlockOwnedDoorsInCell", "Actor", "WillIntimidateSucceed", "Actor" "WornHasKeyword", "Actor"  "IsHostile", "Potion",

There are at least a few dozen more.

» Wed May 16, 2012 2:33 am

Huzzah for extenders! :celebration:

» Wed May 16, 2012 8:46 am

OK - Papyrus (.pex) dumper is done. There's debugging info to allow translation back from opcode offset to original line number, so that might be useful during script development later. Next, there's a list of "user flags", which I would guess are just bitfield assignments for something stored on each class instance. Next we have the object list, which so far as I've seen only contains one entry per file. Each class is defined by a list of member variables (of any type, default values allowed), a list of properties (basically member variables that are backed by code with getter/setter functions), and finally states. States contain lists of functions, and the instance can be switched in to different states to access different implementations of the same function. This sounds really useful for AI/gameplay code - for a guard's AI backed by a class, you could have different states for "patrol" and "arrest" and etc, with a different response to interaction for each. Haven't verified this, but most objects drop everything in an empty unnamed state, so I would assume those functions are always accessible unless overridden by something in a state.

Function opcodes:

Spoiler

kOp_Nop = 0,
kOp_IAdd,
kOp_FAdd,
kOp_ISubtract,
kOp_FSubtract, // 4
kOp_IMultiply,
kOp_FMultiply,
kOp_IDivide,
kOp_FDivide, // 8
kOp_IMod,
kOp_Not,
kOp_INegate,
kOp_FNegate, // C
kOp_Assign,
kOp_Cast,
kOp_CompareEQ,
kOp_CompareLT, // 10
kOp_CompareLTE,
kOp_CompareGT,
kOp_CompareGTE,
kOp_Jump, // 14
kOp_JumpT,
kOp_JumpF,
kOp_CallMethod,
kOp_CallParent, // 18
kOp_CallStatic,
kOp_Return,
kOp_Strcat,
kOp_PropGet, // 1C
kOp_PropSet,
kOp_ArrayCreate,
kOp_ArrayLength,
kOp_ArrayGetElement, // 20
kOp_ArraySetElement,

Interestingly, the opcodes appear to be versioned - array support was only added in the final version shipping with the game (3.1). There's only one in-game script that uses them. JumpT is short for "jump if true", JumpF is "jump if false", the rest of the names seem pretty self-explanatory. Note that there are separate opcodes for integer and floating point math.

edit: Forgot to mention something really important. Functions can be implemented either in the VMScript language, or they can go looking for a native (C++) function for their implementation. It looks like shadeMe has found a bunch of the native functions. Internally they're all wrapped by a templated class describing the function's parameters. It appears that we should be able to start extending functionality by patching the class loader so that any new functions we want are added behind the scenes, then register our own native functions and have the linker take care of the rest. Waaaay less code patching than before; I just need to figure out what all of the virtual functions on BSScript::NativeFunction do (or just do a blind implementation, but I hate doing that).

And, uh, of course wait for the editor to get released so this is actually useful and testable.