Cower.PhysFS - Abstract filesystem access

If that doesn't make sense, I'll explain it this way: with PhysFS, any filesystem access your application has is likely going to be reasonably safer with write-access limited to the directory you specify. In addition to that, you can also access archives, such as ZIP (e.g., PK3 and PK4), Quake 1 PAK files, WAD files, 7zip files, and a few others.

If you've ever worked with the Quake 3 engine, you're probably already familiar with this idea. Personally, I love this sort of system, and think it's a great thing to use. I've written a couple of these myself, but honestly none of them ever reached the level of quality I've seen in PhysicsFS.

If you don't know who this is for, it can pretty much be used by anyone who accesses files in their software. Provided you don't need or want to access files outside of your game's directory (if you do, I'm never downloading your game.. scary) and you want an easy way to add support for archives (they're great for patching game code, by the way), then you can probably make use of this.

At the moment, only the raw API is provided, so you'll want to refer to the documentation on their site (can get there from the link below the download). I'll be working on a wrapper module, but that's going to take some time. In the meantime, you can mess with this.

Also, it should be 100% cross-platform. Can't guarantee it, but the code is there.

DOWNLOAD

Anyway, if you don't know whether or not to use it, you can always head over to their site.

Thanks Noel, this look useful.
Anyhow, good to have you back.

UPDATE

I've updated the archive to include some fixes to the original module as well as including [b]Cower.PhysWrap[b/].

PhysWrap is, as the name suggests, a wrapper for PhysFS that provides easier use. Basically, it adds some functions and stream support via OpenStream using the pfs:: protocol (like incbin:: as mentioned in the documentation).

The code is documented, although you'll have to run docmods/whatever it is in the IDE I think to get it to be included in the documentation. It should more or less explain the few things you have to do in order to get it working.

If anyone has any questions, I'll try to address them, add code, fix bugs, make changes, clarify documentation, etc. as necessary.

The archive now includes these changes and if you're interested in using the module, you should download it. Shouldn't require any changes in code if you've already started using it.

This looks pretty good, but I can't tell how to use it:

Import cower.physwrap
Import cower.physfs

InitPhysFS()

RegisterSearchPath AppDir

Local arr$[]
arr=GlobPhysFSFiles(AppDir,"*")

For s$=EachIn arr
Notify s
Next

Do you have a more, em, specific question?

What do you use for a wildcard? It just crashes if I give it anything other than "*".

I got it to list files in a directory, but it doesn't seem to recognize zip files.

A simple example with a zip file would be great.

First off.. UPDATE

There was a bug in GlobPhysFSFiles, that's been fixed. General issue with dealing with memory.

Now:
When you register AppDir, its contents are added to the root ('/') directory of the PhysFS filesystem. So, it would work like this:

Strict

Import Cower.PhysFS
Import Cower.PhysWrap

InitPhysFS()
RegisterSearchPath(AppDir)

Local arr$[]
arr = GlobPhysFSFiles("/", "*.bmx") ' <- Gets the contents of AppDir

For Local i$ = EachIn arr
    Print i
Next

KillPhysFS()

Input()

As for zip files, you just call RegisterSearchPath("c:\path\to\your.zip"). Of course, it doesn't have to be an absolute path.

If you want to list all the files though, calling Glob is unnecessary as passing a wildcard of * is going to make it call ListPhysFSFiles instead (only takes a directory, and thus lists all the files -- doesn't work the same).

Thanks, it is working now. This is pretty sweet. I love the q3a and Unreal packages, they are very nice to work with (unless you have to write a UPackage reader).

Got any way to test this?

No, I was wrong. It is workign in a simple test. Still trying to figure it out, this stuff is a little different from what I normally deal with.

Hmmm, your code above crashes when not in debug mode.

Unusual. Will look into it, but I'm not promising anything soon.

Well, the issue is with the globbing extension. I'll just rip out my old globbing code from Indigo and add it into this, at least then it'll be fine.

Well, now it's blowing up on the normal file enumeration. Odd.

UPDATE

Alright, everything should be fixed now. The error, somehow or other, was caused by PhysFS being called inside of an Assert during runtime. Now why an Assert would cause PhysFS to flip out and do something odd, or why BlitzMax would do something odd -- bit hard to debug something in release mode, so I can't pinpoint it any better than that it was a problem with Assert being used -- but the problem has been fixed from what I can see.

I've tested it in pretty much every situation I could find it being used in, and thus far I haven't had any problems.

Changes:
* Cower.PhysWrap no longer crashes when in release mode. Something to do with using Assert. I don't know what's wrong there and I can't really say if it's BlitzMax or PhysFS, so I'm at a loss here as to whose side this potential bug is on. (Hell, I can't even confirm if it is a big.)
* The caseSensitive argument has been added to the glob search, but will not affect code as it defaults to the state it was in before I added it (not case sensitive).
* A new exception, TPhysFSException, was added to the wrapper to make up for not using Assert.
* Added the PHYSFS_Allocator type to Cower.PhysFS. This is only useful if you want to write custom memory management code for PhysFS. If you don't, don't bother reading the remaining text.
* Code for an allocator was added to the wrapper source code. While fairly pointless, and commented out at the moment, it should give you a good idea of how to use the setAllocator function if you find yourself needing really customized memory management.

Since BMax and PhysFS both default to using malloc, there isn't much of an issue with memory anyway. I suppose if I was so inclined, I could attempt to hook PhysFS up to the BMax garbage collector, but that would be messy and completely unnecessary.

UPDATE

I've uploaded a new archive, this time 64-bit support is removed from PhysFS until I can find out why it's crashing. If I do, I'll have to then clear it with the folks making PhysFS and see what they think about it and work on getting the changes put into the official codebase if necessary (having to apply my own changes each update isn't something I fancy doing -- there are small changes I make to the code, but large fixes/changes would not be good).

This should not break anyone's code as I highly doubt anyone here uses files larger than 2gb in their game. If you do, sorry, but this is the best I can offer you at the moment.

(Also, I'm pretty sure support for files larger than 2gb was already off-limits in BlitzMax regularly, so probably even less of a loss there.)

Make sure the asserts don't contain any critical code - eg: this is bad:

Assert Graphics( 640,480 )

In release mode, it'll never execute Graphics().

That'd explain it.

Thanks, Mark.

UPDATE

Forgot to remove some debugging stuff. Fixed version uploaded.