Object Serialization Strategy (attn Frank Taylor)

A long time ago I had a discussion with Frank Taylor regarding the "best way" to maintain links between objects. I suggested having fields with pointers to objects, while Frank preferred storing numerical IDs and looking objects up via their ID in arrays. One of his major points was that serialization of objects (storing them on disk and reading them back again) would be very difficult without IDs. I've recently implemented a solution I read about which elegantly solves the problem without resorting to IDs:

When saving objects, store each object's pointer (as a Handle()) and each pointer field simply as a pointer (as a Handle().) When loading objects back in, keep a hash map of old_ptrs to new_ptrs, adding an entry every time you load an object. When you're finished loading all the objects, go back over them and translate their pointer fields using the hash map you built. Voila: the objects now point to each other again!

This works because pointers are guaranteed to be unique. It's even easier to pull off in C++ because you can store a list of pointers to all the pointers you're loading, then loop over them afterwards - although a std::vector<void * *> gives me a headache.

Sort of neat.

The two methods are pretty close actually. Just a difference in how to get a unique ID. I definitely agree with using a hash map to re-establish the links rather than scanning an array or list.

I pretty much use IDs assigned just before serialisation for some reason. Maybe just habit but it works.

Often now when it is a hierarchy rather than a network I don't even use IDs but just structure in an XML file.

Anyway it is a good insight about the memory thing.

The only thing I'd worry about with some languages with automatic GC is that the 'pointers' might change mid operation. That would stuff things up royally.

Hashing is such a wonderful thing. I've been using Defoc8's hash table code in my engine to store a global table of game objects (resources, materials, textures, models, entities, etc.), so I never have to pass stuff by-pointer to Lua, but rather I can just pass the hashed key and get the object from the table. Sounded like the long way around to me, originally, but it's safer since I'm not using pointers to handle script references, and as long as the object is in the table it won't be freed (good thing since I don't want to have to rebuild the VFS/material/texture/font/shader lists [not the fastest thing when you have PAK files you have to scan as well and check for newer versions and so on and so forth] and have to check for validity of scripts and the like -- so instead I store 'stubs' of the resources and they're cached when they get referenced).

I think I just went on rambling in the parenthesses. Anywho, hashing rocks. And no, I don't means smoking.

Would it be extremely naughty to point out that


doesn't quite square with

I'm not obsessing here - just pondering on why flamewars have to destroy potentially useful threads for the sake of a few people flexing their egos online.


Anyway... nice tip octothorpe - nicely compliments your container entries in the CodeArcs... are you thinking of adding this functionality as 'methods' of the containers ?

I've signifigantly modified his code (to the point where it barely resembles the original), so perhaps I should say that I used his design?

And the destruction of the 'whose code do you use' thread was Red Ocktober's fault, not mine. I was only one of the people who said 'mine,' he merely chose to pick on me because, well, I'm me.

Unfortunately this isn't possible in Blitz. Each class (type) of object needs its own method to describe which fields need to be serialized - and there's no way to call a method on an object in Blitz unless you know the exact class (type) it belongs to.

It is possible in C++, and I've come up with a rather sneaky design which only requires me to to have one method per class for both input and output. *glee*