AGI v1 resource extractor and sound converter

[NewRisingSun]

I have written a little extracting utility for AGI v1/v2.001 PC disk images (as well as the usual v2 and v3 volume files), and a converter that converts both GAL, AGI v1 and AGI v2/v3 sound files to VGM format for convenient listening. For later v2 and v3 games, you need to specify the --envelopes2 or --envelopes3 command line option to get the same result as in the games, as AGIsnd2vgm has no way of knowing whether envelopes should be applied (AGI v2.9xx) or not (AGI v2.4xx/v2.272/v2.089), and which shape the envelopes should have. Run AGIsnd2vgm without options or file names to see a list of supported options. The "stuck notes" option can best be seen action in King's Quest II's SND.1, which is the wedding music.

I have noticed that AGI v1 files that were converted by Sierra to v2 format sound faster when played back in the latter format. For example, the music in The Black Cauldron versions 2.0 and 2.10 sounds faster than in v1.1J/K/M. The reason is that the playback routine for the v1 format used the PC's default 18.2 MHz timer, while AGI v2 uses a 60 Hz timer, and Sierra's CNVRTSND.EXE multiplies all time values only by three instead of the correct 3.2967. CNVRTSND.EXE is Sierra's utility to convert v1 to v2 format files and what accidently included on the Space Quest I version 1.1A disk #1.

[lance.ewing]

I'm going to take a look at this this evening.

I find it interesting that the CNVRTSND.EXE utility was on a Space Quest 1 disk. Does it appear as an undeleted file?

If so, it makes me wonder if Space Quest 1 began life as an AGI v1 game but was converted to AGI v2 at some point prior to release.

[lskovlun]

If so, it makes me wonder if Space Quest 1 began life as an AGI v1 game but was converted to AGI v2 at some point prior to release.

Certainly, this occurred on other (later) occasions. LSL7 Demo is SCI2.1, while the final game is SCI3, suggesting it was made in SCI2.1. I recall getting quite far in KQ1 SCI with pre-Glutton FreeSCI (which had the same heap limits as SCI0), suggesting they weren't deliberately taking advantage of the extra memory provided in SCI01. There may be other instances that I'm forgetting.

In both of these cases, porting may have been a simple matter of recompiling the source.

[Collector]

Other demos with earlier interpreters than the finished game are GK1 demo SCI1.1, finished game is SCI2. PQ4 demo SCI1.1, finished game is SCI2. RAMA Demo 2.1, finished game is SCI3. Shivers Demo SCI2.1, finished game is SCI3. Lighthouse Demo SCI2.1, finished game is SCI3.

[NewRisingSun]

Updated my AGI tools (edit: updated again!):

• Having found out that AGI v2.9xx and above use different note envelopes on the IBM PCjr versus Tandy 1000, AGIsnd2vgm now supports both and allows specifying the desired type on the command line (--pcjrenv vs. --tandyenv).
• AGIextract now creates text files that can be used to rebuild the VOL files. They are named VOL(number).DRV in the case of AGI v1/2 and (gameId)VOL(number).DRV in the case of AGI v3 games. The name and format has been chosen to be suitable for use with Sierra's MKVOL.
• AGIdecompile is my new customizable AGI logic decompiler. It requires
  • ..\WORDS.TOK in AGI v2 format. When decompiling an AGI v1 game, take that one file from the respective AGI v2 version of the game you're trying to decompile. This limitation will be removed in a future version. The "../WORDS" file for AGI v1 games, or the "../WORDS.TOK" file for AGI v2 games, which is automatically extracted by AGIextract.
  • GAMEDEFS (that %includes SYSDEFS) in the current directory. Depending on whether you are decompiling an AGI v1 or v2 game, copy template/src/SYSDEFS.v1 or template/src/SYSDEFS.v2 to SYSDEFS.
  Sources are always output to ../src/. AGIdecompile is designed to produce source files that Sierra's CG (of an appropriate version) will compile without complaining. This implies that messages have to be put separately and referenced by number, rather than being specified in-line the way AGI Studio uses them. Whenever AGIdecompile outputs a GOTO, it adds a comment to explain why it resolved the FE jump instruction as a GOTO instead of another ELSE brace. Since AGIdecompile itself does not know which interpreter version to decompile for (it's all defined in SYSDEFS), for AGI v1, variables and flags should be specified with identical names in GAMEDEFS/SYSDEFS. SYSDEFS.v1 now defines all commands in such a way that there are only flags, not variables.

I have successfuly extracted, decompiled, then recompiled and repackaged the game PC KQ2 v2.2. The source codes of these utilities have become a bit messy. I will clean them up (possibly rewrite in C++?) once I know what the final functionality will be and once more knowledge about AGIv1 in particular has been acquired.

[OmerMor]

Well done!!!

When you decompile and recompile, do you get exactly the same bytecode?

[NewRisingSun]

There are two systematic differences:

• My decompiler outputs the final return(). CG expects source files to have no return() at the end, and always adds its own. Therefore, when recompiling files produced by AGIdecompile, the recompiled files have two returns at the end. I should modify AGIdecompile to suppress the final return().
• The original logic files sometimes have FE 00 00, which means jump +0, which has absolutely no effect. AGIdecompile ignores such instructions, as they may confuse my resolving of jumps to ELSE braces. But when CG recompiles those source files, the recompiled logics don't have the FE 00 00 either. So I wonder how they got there in the first place, since obviously they are not generated by CG. I could imagine the programmer inserting an explicit GOTO to a label that immediately follows the GOTO, for example because there used to be code in-between that was removed at some point. This occurs several times in BC.

There are eight logics in KQ2 v2.2 with other differences that I still need to investigate. When I decompile these recompiled logics again, the sources are completely identical, for what it's worth.

Also, CG does not compile a few sources at all but exits with an "output buffer overflow", notably PQ's rm0. That happens exclusively with late 1987 games, while my CG.EXE has a late 1986 time stamp. Either I need to specify a command-line option that enlarges the "output buffer", or the later games need a newer CG.EXE. I don't think that's the fault of the decompiler though.

[NewRisingSun]

After having rewritten the ELSE/GOTO handling at least five times, AGIdecompile now produces sources that when fed to CG now yield 100% identical byte codes. I have tried this successfully with KQ3 v2.14, SQ v1.0X, KQ2 v2.2., and Donald Duck's Playground v1.0Q. SQ v1.0x (AGI v2.089) and DP require modifying SYSDEFS so that quit() has no argument, and DP v1.0Q (AGI v2.001) additionally requires set.game.id to have a NUM instead of a MSGNUM argument.

The version of CG that I have (v3.14) so far refuses to compile AGI v1 scripts at all; complaining that it does not know the "==" etc. operators. How strange.

[Kawa]

100% identical byte codes

Quoting this for well-earned emphasis.

[NewRisingSun]

Figured out how to prevent CG from giving the "output buffer overflow" error. I had used the -b parameter before, but with values that were too high or too low. CG likes values between 4096 and 8192, and the value might not be the same for every script.

With that in mind, I have managed to decompile and recompile AGI v3 logics as well. I had never noticed that AGI v3 games usually do not encrypt the message strings with "Avis Durgan", probably because the LZW compression makes it unnecessary. Most logics don't have encrypted messages, a few do, probably the ones that are so small that LZW compression isn't used. I had to modify CG.EXE by replacing "Avis Durgan" with a zero string so I could compare the original and recompiled bytecode properly. Added support for non-encrypted messages to AGIdecompile. Also added the additional AGI v3 commands to SYSDEFS.v2.

Therefore, I managed to decompile and recompile with identical results:

• Donald Duck's Playground v1.0Q, AGI v2.001
• Gold Rush v2.01, AGI v3.002.149
• King's Quest I v1.0U, AGI v2.272
• King's Quest II v2.2, AGI v2.426
• King's Quest III v1.01, AGI v2.272
• Leisure Suit Larry v1.0, AGI v2.440
• Mixed-up Mother Goose (no version number), AGI v2.915
• Manhunter New York v1.22, AGI v3.002.107 (Has one bad jump instruction in RM.159, jumps into the middle of an IF block?)
• Police Quest v2.0G, AGI 2.917
• Space Quest v1.0X, AGI 2.089
• Space Quest II v2.0F, AGI v2.936
• King's Quest IV v2.0: has strange jumps in rm0 that causes AGIdecompile to lose sight of its braces, and the AGI interpreter to crash or behave erratically when CTRL+R is pressed in CGA mode. As this does not occur in v2.3 of the game, I attribute this entirely to an error in the original bytecode. CG v3.14 freezes upon compiling several of the decompiled KQ4 scripts apparently because of their very long said strings (five to six words). But those that do compile are identical to their originals as well.

Now, to figure out what's going on with CG and AGIv1 sources.

[Collector]

Did CG.EXE ever get posted here?

[OmerMor]

Now it has.

[Collector]

Thanks.