Object storage 'framework' using SQLite

BlitzMax Forums/BlitzMax Programming/Object storage 'framework' using SQLite

Hi I'm just working on a simple object storage framework, using Bruceys SQLLite module...

The aim is to allow any object and it's fields to be saved and loaded from a SQLite database, using reflection to obtain the data.

The project is just starting out but wondering if anyone has done something similar?

Or can think of good test cases/pitfalls that should be factored in?

Brucey did persistence.mod which stores the data in an XML file. Which is, I think, a lot easier to do then using a SQLite database... At least in the case of serialization.

So why not use that?

@Htbaa Can't find a link to persistence.mod in Bruceys modules...

I liked the idea of a database, more modular, and SQLite allows you to use a single flat file...

The plan is to move all of a games media content into a single database file along with the game's object data with the aim of a more data driven approach to development! ;o)

And depending on speed I could also store hi-scores, statistics and even basic event logging!

Mainly I haven't done much with databases for a while and always found XML to be a bit overblown for most uses!

Merx, the persistence module (and pretty much all the other new modules) are on the maxmods Google code project.

http://code.google.com/p/maxmods/wiki/PersistenceModule (it is currently only in the svn repository)

But wouldn't you need blobs for that - which were missing last I looked?

Yes I will but I've checked out the SQLite interface for Blobs and it allows you to extract chunks of a Blob at a time, so it's probably best to use a Bank for processing in a two pass approach, get the size of the blob, create a bank of that size and pass in the bank to be filled!

It should be simple enough once, I get my head around C/Blitzmax conversions, the fun part will be integrating/hacking it into the current module.

Unless anyone has a better idea?

PS> I have only got it up to the stage of saving simple objects with int, double and string fields so far!

Next relaoding objects...!

Yeah it now saves and loads basic objects [objects with int,double,string fields]!

Cuppa then need to get linked objects to save and load!

If you can think of any tests cases that will enhance this let me know?

I will post the code shortly, hopefully I can improve it and enhance it if the community is interested?

I considered doing this for my current app - so, yes seeing what you come up with would be great thanks.

Brucey did persistence.mod which stores the data in an XML file. Which is, I think, a lot easier to do then using a SQLite database... At least in the case of serialization.


That very much depends on what you are trying to achieve. And ORM (Object Relational Mapper) like http://www.sqlobject.org/ (for python) as an example abstracts away lots of normal SQL-CRUD (Create Read Update Delete) while at the same time still having the benefit of fast searches from the database.

If you have 10 million records/objects and you need to look up and alter 100 of them for some processing an XML file serialization is clearly a worse solution than ORM functionality.

ORMs often allow you to either define classes and it will create an table-structure in the database matching these classes or even vice versa.

ORMs are also often quite smart in that if a class has an internal list of other objects (i.e. in database terms - a relation), the ORM often attempts to lazy-read the "sub-objects", i.e. they are looked up and accessed first if/when the program interacts with the SUB-list. This functionality uses so called "proxies" in several languages that supports them. A proxy is a function that is called instead of any function call of a class, to provide extended functionality. The proxy can then, in turn, evaluate the call and call internal functions in the class to get the actual job done.

Sorry for long text. TL;DR version:
Serialization <> ORM (different uses)

Hi Mahan,

I'm going for more of a Serialization solution to database the aim is to have a games complete dataset in a flatfile sqlite database and to load all the data from that.

Hopefully this will also allow for moving to a more data driven development approach, so with a simple object editor tool you can update and amend the data, in effect altering the game without having to recode!

An ORM sounds cool but most games go through loading/saving phases and I would like to keep this simple to start with!