It is well known that IDs of Forms added by mods will change based on the load order. But what I didn't know was that any forms that are not based on a pre-existing form will disappear altogether when the load order is changed.

Currently I fixed this problem by checking if any of my references have gone missing and re-add them if they have. But is that the right way to fix it? How do other mods handle this kind of problem?

The load indices are all that change when a plugin is loaded earlier or later than it previously was, thus loading a new mod (regardless of load order) doesn't break one's save such that all the other previously loaded plugins are seen as new or anything. The save stores the data relative to to the plugin, so a plugin's load index changing will effectively not adversely affect the user. A YourNiftyItem reference, whether placed in the Creation Kit manually or created by a script, will remain in place even if plugins are added/removed/reordered.

It's possible however, if a plugin has multiple masters and the masters' load orders are inverted relative to the header's master list, for reference info to get broken. When all the DLCs are out, we should all be playing/creating mods with them loading in the same order to prevent this happening to plugins with multiple DLCs as masters. Then, if multiple mods are loaded which both have dependencies upon multiple DLCs, reference info will line up in both cases as their masterlists will be in line with the actual load order.

For instance, PluginY.esp could be in trouble when referencing forms from either DLC if its masterlist is out of order while PluginX.esp's references will work as intended:
[00] Skyrim.ESM
[01] Update.ESM
[02] Dawnguard.ESM
[03] Hearthfire.ESM
[04] Killer Bees.ESM
[05] PluginX.esp <--- Has Skyrim.ESM, Update.ESM, Dawnguard.ESM, Hearthfire.ESM, and Killer Bees.ESM as masters in that order
[06] PluginY.esp <--- Has Skyrim.ESM, KillerBees.ESM, Hearthfire.ESM, and Dawnguard.ESM as masters in that order

What makes the most sense, IMO, is to load them in chronological order of release, regardless of the order of acquisition.

It seems you are right, I can no longer reproduce the bug, which makes me wonder how my custom object keep disappearing from the inventory for no apparent reason when I'm just working on the scripting for the mod.

Update: I found the problem, a very odd one: After replacing my mod files I need to edit the load order for the items to appear, but the order doesn't matter. I can't think of any reason why this happens though.

Sounds like a conflict with something else that's loaded.

I just can't seem to reproduce the problem consistently, it just seems random. I'll just continue modding and see when it happens again, maybe it's just a problem with my mod. Thank you for the help, it taught more about load orders and removed any doubts I had about the safety of changing Form IDs due to load order.

I triggered the bug again, but I can't seem to resolve it this time. I'm very tempted to just check for invalid references when my script is initialised.

What happens is the custom object just disappears from my inventory no matter what save I load or what version of my mod I use. I compared all versions of my mod with TESVSnip and none of them has a different Form ID for the custom object. Re-adding the object works fine.

Are you using TESVSnip for anything but viewing data? It's known to have issues.

Only viewing data, nothing more.

I tried disabling all other mods and putting mine at the top of the load order, still nothing. I know the mod is loaded correctly, the scripts are active and I can find my custom object using the help command.

When I re-add my custom object and make a new save I can load that save without the item disappearing.