Resource Dumper & Creator

[gumby]

Based on a suggestion by Collector, I've begun a project to extract (and later merge) all resources from a RESOURCE.XXX file, putting them in their own individual files.

I have determined all the different resource types - I'm sure this is documented somewhere, but I couldn't readily put my hands on it.  I was able to get this information through cross-referencing in Companion.

0 - View
1 - Pic
2 - Script
3 - Text
4 - Sound
5 - unused?
6 - Vocab
7 - Font
8 - Cursor
9 - Patch


Anyway, I'm half done with the tool, the 'dumping' part.  I stole the vast majority of the code from my previous textDumper tool.  Hopefully the rest will be competed soon.

[Collector]

I am assuming that you are only working on it for SCI0. Later versions (SCI1? and later) use a different naming convention, i.e. *.SCR instead of SCRIPT.*, instead of PIC.* and *.V56 instead of VIEW.*, etc. You also get additional resources that don't exist in SCI0.

Code: [Select]


Resource SCI0       LSL 1.000.011     SCI1   Later SCI32

0 - View VIEW.* *.V16 *.V56
1 - Picture PIC.* *.P16 *.P56
2 - Script SCRIPT.* *.SCR *.SCR *.CSC
3 - Text TEXT.* *.TEX *.TEX
4 - Sound SOUND.* *.SND *.SND
5 - unused?
6 - Vocab VOCAB.* *.VOC *.VOC
7 - Font FONT.* *.FON *.FON
8 - Cursor CURSOR.* *.CUR *.CUR
9 - Patch PATCH.* *.PAT

? - Pallet *.PAL
? - Audio *.AUD
? - Message *.MSG
? - Heap *.HEP
? - Wave *.WAV
? - Clut *.CLU
65535.MAP - Map file for the sound effects file, RESOURCE.SFX

AUDIO36 and SYNC36 have a less defined naming convention for the resources packed in the RESOURCE.AUD file.

Sequence files (*.SEQ) are not packed in the RESOURCE.*, so do not have a MAP file. The same goes for the movie files (*.VMD, *.DUK).


I have included the LSL 1.000.011 as it is a little different. Specifically, I am trying to add a
patch to it. Unlike most, patch files placed in the game's folder are ignored and only the resources packed in the RESOURCE.* are used. I found this:

It is not possible to get early SCI0 interpreters to take notice of a SCRIPT.XXX file in the directory. If patching the existing resource files is too much work, you can put your patched script resource into an additional RESOURCE.XXX file (say RESOURCE.010) and modify that one entry in RESOURCE.MAP accordingly.

I am looking for a way to do this so I can add the patch to my installer and without having to include a modified version of an existing RESOURCE.* file. Additionally, both studio and Companion refuse to open this version.

[gumby]

Yes, SCI0 only - at least at this point.

I was wondering if one could put SCRIPT.XXX files in the same directory & have the game use them (like PIC and VIEW files).  That answers that.

Collector, I understand what you are driving at.  Instead of modifying an existing RESOURCE.XXX, with my implementation you should be able to create a whole new RESOURCE.XXX file (with just the one patch resource in it) and then add it to the map.  Should not be a problem.

For a naming convention when dumping out resources, I'm leaning towards creating a subdirectory for each resource file number, like this:

001
   SCRIPT.001
   VIEW.002

002
   SCRIPT.100
   SCRIPT.101
   SOUND.002

...etc.  So when the resource 'build' step occurs, one would point it at a directory tree containing files/folders with this naming convention & it would pack them up into new RESOURCE files.  It will not append to existing resource files - this will be a build-from-scratch process.  In the above example, two resource files would be created, RESOURCE.001 and RESOURCE.002.

Essentially, one must dump all the resources (unless you are creating a new resource, as NewRisingSun referred to) first & then make your modifications to the files you want, then it will pack them all up for you.

[Collector]

I was wondering if one could put SCRIPT.XXX files in the same directory & have the game use them (like PIC and VIEW files).  That answers that.

And that is the problem. The early SCI0 games ignore the resources in the game's folder. Most SCI games do use resources in the game's folder or in the path listed in the RESOURCE.CFG. This made it easy for Sierra to distribute patches because they could be applied by simply copying the updated raw resources in the game's folder or in the path listed in the RESOURCE.CFG and the game would use those over the ones packed in the RESOURCE.* and specified by the MAP file.

What is odd about this version of LSL2 is that the naming convention seems to be halfway between the two (PIC.* vs *.P16 vs *.P56), which suggests that it is not the early SCI0 like the KQ4 in question in the NRS reference (version 1.000.111 with Interpreter 0.000.274). The LSL2 version 1.000.011 has interpreter 0.000.343, but is ignoring patch files just like the early SCI0 does.

[gumby]

An interesting development:  RESOURCE.XXX files support compressed resources & 'patch' resources do not ('patch' here meaning resources external to the RESOURCE.XXX file - for example VIEW.001 or PIC.004).

Since we are starting from RESOURCE.XXX files, dumping out the entire contents & then rebuilding them (presumably after some manipulation of the individual resources), we are presented with potentially non-reproducible RESOURCE.XXX files.

Let me elaborate.  Say you start out with a RESOURCE.XXX file that contains compressed resources.  Dumping them out renders them uncompressed.  When we attempt to 'rebuild' the RESOURCE.XXX file from individual 'patch' resources, we have no indication as to how they may have been originally stored in the RESOURCE.XXX file that they were dumped from.

In the interest of saving time, I'm not going to perform any compression when rebuilding the RESOURCE.XXX files.  This is unfortunate from a validation standpoint; it would be dead-simple to compare a newly rebuilt RESOURCE.XXX files to the original, assuming no changes were made and that the compression methods remained intact.  I suppose when dumping the resources, I could retain the original compression method in a meta-file somewhere & then look it up at the time of the rebuild.  I would like to implement LZW encoding (as I've mentioned in the past w/regard to text resource compression), but I just don't have the time to do that right now.  Oh well.

[gumby]

Feeling a bit silly right now.  I absolutely need to account for the compression type because when I dump the resources, despite the fact that compressed external patch resources are not supported, I have not decompressed them.  Which means that if someone wanted to use these dumped resources in a game without re-packing them, they wouldn't work.

Collector - is it possible that this is the problem with your external patch file?  Do you know if it's compressed?

So I'll either need to to decompress the resources as they are extracted, or keep track of the compression method somehow.

I've got the utility unpacking & packing, with byte counts matching up - no further validation yet.  Creates multiple subdirectories, one for each resource.xxx file.  The packing works on all subdirectories, creating a resource.xxx file for each - the reverse of the dump.  Looking good so far.

[Collector]

It is not compressed. Check your PM on SHP.

[gumby]

Okay, this tool is complete.  I've uploaded it here (via the game upload) & to the wiki.

The major limitation with this tool is that it does not support compressing or decompressing resources, so from a 'dump' perspective it's really only useful for games made in Studio or Companion - no Sierra games.

This tool would also be useful for Sierra games for the purpose of packing patch files into resource files.  Collector, I believe this is the functionality you were looking for.  The only problem I see is that after packing the new resource you will need to rebuild the map file.  Currently my recreateMapFile tool does not account for the possibility of duplicate resources.  I'm pretty sure you'll have to hack the map file manually.