Sierra's Internal SCI Tools

[lance.ewing]

Call me weird, but that sounds EXACTLY like my kind of jam. I'm all for this. Go big or go home.

troflip was probably half joking, but I've also thought the same thing a few times. A year or so ago, we even identified what "IDE" they were using back then and found a site online that appeared to have a version of it that roughly matched what Sierra would have been using. This was based on a partial match between a fragment in an unused part of an original game disk with one of the standard BRIEF macro files that comes with BRIEF, and based on the publish date of that particular version of BRIEF.

One thing I haven't heard of turning up yet though are the custom BRIEF macros that Sierra used. They clearly used a lot of them. They are referenced in some of the documentation that Omer released, and we've seen a few fragments from original disk slack space, but I don't think anyone has any full macro files.

The AGDS document included in the AGI Original Documentation thread mentioned BRIEF a number of times. Here is an interesting bit:

Code: [Select]

CG -- the game compiler.
This is the most used. Assuming you use BRIEF to edit the source code files,
call the compile macro (often assigned to the F9 key). It saves your file,
calls the CG game compiler, routes CG's output to your LGC subdirectory, and
if errors are found, displays them in the current window.

...and this bit as well:

Code: [Select]

MACROS
The BRIEF macros "allfiles.m" and "modify.m" allow you to modify all of
your source files at once. To use them, edit the file "\brief\macro\modify.m"
to do whatever you want done to your files. Compile it (use F9) until it iscorrect, and automatically loaded.
enter "allfiles."


[Kawa]

Me, I just want compression and maybe multiple volume files. But I can't get MAKEVOLS to work.

[lskovlun]

Me, I just want compression and maybe multiple volume files. But I can't get MAKEVOLS to work.

This reminds me of how the first decompressor I wrote for SCI1.1 abused a command-line tool from Logitech that happened to use the same compression library. Christoph's first comment to that was, predictably, "WTF?"
But it worked.

[NewRisingSun]

Me, I just want compression and maybe multiple volume files. But I can't get MAKEVOLS to work.

I did get it to work to rebuild Space Quest I with compression. What exactly is your problem, and what version of MAKEVOLS are you using, and for what SCI version are you building?

This reminds me of how the first decompressor I wrote for SCI1.1 abused a command-line tool from Logitech that happened to use the same compression library. Christoph's first comment to that was, predictably, "WTF?" But it worked.

Ah, ye olde PKWare Data Compression Library. I first extracted it from SIERRA.EXE to create a linkable library version of it, then debugged it and wrote my own implementation of the decompressor using assembly language. Of course, with the C implementations available in open-source, that effort has become useless. I'm not positive though whether the compressor has ever been rewritten in Assembly or C as well, or whether an official linkable library has ever been leaked. PKWDCL seems to have been used quite widely --- not only did Sierra use it, but LucasArts would distribute Monkey Island 2 and Indiana Jones 4 using their own custom archive format (LFG) that compressed using the library. And many mid-90s floppy-based games used the INSTALIT installation package. Its built-in decompressor first used the old PAK compression from NoGate Consulting but later switched to PKWDCL in later versions. I know that from writing my own InstalIt .PVL archive extractor.

[Kawa]

I did get it to work to rebuild Space Quest I with compression. What exactly is your problem, and what version of MAKEVOLS are you using, and for what SCI version are you building?

The one from OmerMor's SCI16 release, and the one from OmerMor's SCI16 release. The problem: I know now what RESOURCE.TXT should be, but MAKEVOLS wants a WHERE file. I have one of those, adjusted to the best of my knowledge but all I get is a blank RESOURCE.000 and RESOURCE.MAP. I'll try again so I can document the error after dinner, which is now.

Given the following:
WHERE

Code: [Select]

vocab=c:
script=c:
text=c:
heap=c:
message=c:
videoDrv=c:\vga320.drv
soundDrv=c:\std.drv
kbdDrv=c:\ibmkbd.drv
joyDrv= NO
mouse = YES
minHunk=1k
pic=.;c:\*.p56;c:
view=.;c:\*.v56;..\art\view\*.v56;c:
palette=.;c:
font=.;c:
sound=.;c:
RESOURCES.TXT
(Loosely based on http://symphoniae.com/nrs/sciprogramming/RESOURCE.TXT)

Code: [Select]

(volumes
volume 0
script 255
heap 255

script 0
script 10
script 11
script 14
script 15
script 20
script 21
script 24
script 26
script 100
)
Result
(Running this from DOSBox, where C: is my game's folder, with all resources extracted.)

Code: [Select]

MAKEVOLS 3.01 Copyright Sierra On-Line, Inc., Apr 27 1993. All rights reserved.

MAKEVOLS: (2) Object (volume) not found.
MAKEVOLS: (3) Object (umes) not found.
MAKEVOLS: (3) End of file while building (null).
resource.000 is 0 bytes long.
Bytes read: 0
Bytes written: 0
Compression: 100%
Sorting...

Resource.000: 0 bytes
Resource.map: 7 bytes

Removing particular script lines from resource.txt yields a shitton of Object (olume) not found. errors.

[OmerMor]

One thing I haven't heard of turning up yet though are the custom BRIEF macros that Sierra used. They clearly used a lot of them. They are referenced in some of the documentation that Omer released, and we've seen a few fragments from original disk slack space, but I don't think anyone has any full macro files.

Here's an archive with BRIEF, some utilities, and compiled macros I managed to scrape off my stuff:
https://drive.google.com/file/d/0B5j-_ZMS8_UoYVJIVEtnZk5lbWc/view?usp=sharing&resourcekey=0-qPC6H-efAXoDFphQ7nquyQ

[NewRisingSun]

I have just rebuilt KQ6 floppy using the MAKEVOLS utility from OmerMor's SCI16 archive. The resulting RESOURCE.000 and RESOURCE.MAP are not byte-for-byte identical, as the MAKEVOL.EXE seems to be of a later version that inserts an additional 0x12 0x34 0x00 0x00 into RESOURCE.MAP. A few of the files also compress differently, indicating a different version of the PKWare Data Compression Library. But the result still works nicely with the original interpreter included with the game. Attached find my WHERE and RESOURCE.TXT file for KQ6 Floppy. I put my individual game resources are in a subdirectory named INPUT\ under the current directory from which MAKEVOLS is launched.

[Kawa]

That works much better!

[NewRisingSun]

MAKEVOLS is actually quite flexible, which in the case of SCI0/10 with multiple volumes is very useful. Basically, you can have as many lists as you want, with one list calling up another. For example, you could do the following (stylized example):

Code: [Select]

(CDROM ; all resources in one volume file
system
rm1
rm2
)

(360K
volume 0
system

volume 1
rm1

volume 2
rm2
)

(720K
volume 0
system

volume 1
rm1
rm2
)

(system
script 999
script 998
)

(rm1
script 1
pic 1
)

(rm2
script 2
pic 2
)Anything that is not a recognized keyword is interpreted as a reference to a separate list. When MAKEVOLS is run without a list name, it just takes the first one in RESOURCE.TXT. If you specified "MAKEVOLS 360k" with the RESOURCE.TXT above, only the "360K" list would be built, which puts all resources associated with rm1 and rm2 in separate volumes 1 and 2, whereas "MAKEVOLS 720K" would put them in the same volume 1. That flexibility still exists in MAKEVOLS 3.01, though it is somewhat useless since that version of MAKEVOLS/SCI does not support multiple volumes anymore.

Some early versions of MAKEVOLS, but not 1.10 and not 3.xx, apparently specified the paths in RESOURCE.TXT as well using the "resource" keyword (from INN):

Code: [Select]

resource view     0x80 /app24/ll/view/%d.v56;/app24/appshare/view/%d.v56
It seems that the paths were moved out of RESOURCE.TXT into the WHERE file at some point in time, possibly because SCI.EXE (without V) requires a WHERE file anyway which could be reused.

[Kawa]

Haha. "Unable to allocate inBuffer", try again, "unable to allocate work buffer" X3

[lskovlun]

The resulting RESOURCE.000 and RESOURCE.MAP are not byte-for-byte identical, as the MAKEVOL.EXE seems to be of a later version that inserts an additional 0x12 0x34 0x00 0x00 into RESOURCE.MAP.

That very much sounds like it's meant for the version-stamping utility to find and change... the thing is that there is a version stamp in the resource map and the executable that must match, but they differ by 0x1234. I helped troflip handle this for the LSL6 interpreter, and I believe it has a base stamp of 360 (dec), so the resource map must have 360+0x1234=0x139c.