Original AGI Interpreter

[lance.ewing]

*Split from the "C# AGILE" topic*

I reassessed the amount of original AGI source code that we have, stating now in the github repo README that it is 80%. My original estimate was based on the number of files, which is still calculated to be about 75%, with the inclusion of the extra source files from the KQ3 disk, but if we look instead at the actual amount of code across all modules, then it works out to be 82%. I rounded that down to 80% in the readme, as I didn't want to be too exact, as it probably depends a little on what exactly you count.

It looks like I should be able to increase the stated percentage even further, although I'll need to work out by how much. The reason for being able to increase it is due to some of the modules being from the library files of the Mark Williams C compiler, or from the PLINK86 OVERLAY.LIB library, which means that they're not technically part of the original AGI interpreter source. So far I've identified the OVLYM module as being the Overlay Manager module from PLINK86, and the SETJMP and PRINTF modules being from the Mark Williams libraries. I did wonder about the FILEIO module, but I've seen evidence in the slack space of some original Sierra disks that FILEIO.OBJ was a file that they compiled from source, i.e. not from a LIB file. I can't spot any other modules in the list that may have come from a standard library, so I think OVLYM, SETJMP and PRINTF are the only ones I can ignore for the purposes of determining the percentage of missing AGI source code.

Edit: The percentage didn't end up increasing that much. Ignoring those three modules, the percentage of original AGI interpreter source code that we have is approximately 83.5%.

[OmerMor]

I did wonder about the FILEIO module, but I've seen evidence in the slack space of some original Sierra disks that FILEIO.OBJ was a file that they compiled from source, i.e. not from a LIB file.

Have you looked into the FILEIO source from the SCI interpreter? Perhaps it's similar.

[lance.ewing]

Thanks for the tip Omer. I hadn't looked in the SCI interpreter source yet. Looking over it, there are many similarly named functions in there, so I think the file/module serves a similar purpose. Its not an exact match though. As the original AGI FILEIO module appears to have been an ASM module, then it should be possible to reconstruct it by hand, assuming the actual source doesn't turn up. This is what the PLINK86 memory map has for the FILEIO module:

Code: [Select]

    Module A from file FILEIO
        Segment CODE.CODE, Addr = 58EC, Size = 1EE
            CLOSE                                                 0:5991
            CREAT                                                 0:58EC
            DUPLICATE                                             0:59D3
            FILEDT                                                0:5AB2
            FINDFIRST                                             0:5A40
            FINDNEXT                                              0:5A65
            GETCURDIR                                             0:59F1
            GETCURDRIVE                                           0:5A29
            LSEEK                                                 0:59AA
            OPEN                                                  0:590D
            READ                                                  0:592E
            UNLINK                                                0:5974
            VALIDDRIVE                                            0:5A7D
            WRITE                                                 0:5951
        Segment DATA.DATA, Addr = AFA1, Size = 0


[Kawa]

SCI's FILEIO is also done in assembly though.

[lance.ewing]

Yeah, I know. Sorry, for the confusion. I think I worded my post poorly. Rather than implying that the SCI FILEIO module was not ASM, what I meant was that since the AGI FILEIO module appears to have been an ASM module (deduced from the AGI.MAP file, which specifies the module name as "A", which appears to be used as the module name for all ASM modules in the AGI.MAP File), then it should be possible to use a disassembly of a similar AGI version (e.g. 2.903) to come up with an ASM module that looks roughly like what the original ASM source file looked like. Even the word "possible" is a bit poorly used here though, since obviously we could use a disassembly to deduce what the C code looked like, if FILEIO had been a C module. So what I really meant to convey was that it would be easier (rather than possible) to put together what the original FILEIO.ASM module looked like in AGI, given that it was an ASM module.

[Kawa]

Thing is, SCI's FILEIO module is more or less a minimal implementation of the old C library's file I/O functions. Before fopen(char* filename, char* mode), we had just open(char* filename, int mode). They're so standard, the actual open and read functions in SCI aren't even in FILEIO.S but in STDIOASM.S! With a name like that...

Combine the two and you should be able to reconstruct a single assembly module with all the stuff you listed.

[lance.ewing]

That does make me feel like they got the code from a standard C library rather than writing it themselves. The Mark Williams Compiler versions 3 and 4 that I looked at didn't appear to be an exact match, although they do have a lot of those functions, just not all of them in a single module, and some I couldn't find at all. I can't say what the earlier versions of the MWC compiler had though, and there is evidence in floppy disk slack space that they were compiling with a v2.3.8 version of MWC at some point. Maybe that had a closer match.

Also in the floppy disk slack space of some original game disks are fragments of a DOS directory table that includes FILEIO.OBJ (alongside other .OBJ files known to be part of AGI), which initially suggested to me that they compiled it themselves, since I thought that the compiler would have something like that in a .LIB library file. Now that I've said that though, I have noticed that the v3 and v4 versions of MWC that I have both have .OBJ files in their LIB folder, in addition to the .LIB files. Perhaps the older versions of MWC had the FILEIO.OBJ file separate but it was then split up, refactored a bit, and included into a LIB file in the v3/v4 versions.

Interestingly, the FILEIO.OBJ file is immediately followed by VGGRAPHX.OBJ in the directory table fragment. That VGGRAPHX module wasn't added into AGI until the 2.9XX versions of AGI. The decoded timestamp from the directory entry for VGGRAPHX.OBJ is the 03-Sept-1987, and the decoded timestamp for FILEIO.OBJ is 25-Aug-1987. The origin of this directory table fragment must have been a hard disk, which seems to be the case for a lot of the slack space of floppy disks, since the cluster number is way beyond the size of a floppy. As I understand it from reading up on this years ago, the slack space data came from data left over in the buffers that DOS (or the copy tool; can't remember which) used when it copies the files to the floppy. So its an interesting view on what Sierra were using on their development machines. In the example I'm looking at, the original game disk is a 2.0C version of SQ2, and the hard disk appears to have had WHERE.SQ2 on it, immediately alongside these .OBJ files. The WHERE file is used with the test/dev non-production version of the AGI EXE. This would have been a WHERE file for SQ2 then.

This thread has gone way off the topic of the C# version of AGILE, but the main culprit is me. I wonder if we can split the recent unrelated posts off into a thread related to the original AGI interpreter source.

[Kawa]

To be honest, if you grabbed a copy of say Turbo C or whatever, wrote a program that uses open and such from the standard library, and looked at the resulting binary, you'd probably see the same thing as in FILEIO.S: that they're fairly thin interrupt calls to DOS. fopen is much more involved.

[russdanner]

Wow, you guys are bit-twiddle super sleuthing here. Impressive!

[lance.ewing]

Does anyone know what assembler in 1987 would have been used by Sierra with the .ASM files in this folder:

https://github.com/lanceewing/agi/tree/main/src

e.g.

https://github.com/lanceewing/agi/blob/main/src/RANDOM.ASM

That RANDOM one is a very simple example. I assumed that they used MASM, and indeed some of the simpler files (e.g. JRJMPTBL.ASM) do work with MASM, but most of them fail with syntax errors. I've tried MASM 4 and 5. They complain about the .if, .else, .end, enter, exit, return, do, dloop, and a few others. I search online and can't seem to find anything that supports all of these. I'm starting to wonder if they had some custom macros defined. They did have a macros file, and it is included in files such as the RANDOM.ASM one, but the version of the macro.ah file that I extracted from the KQ3 2.14 disk doesn't have anything to support those.

[mnicolella]

This page suggests that those directives were added in MASM 6.0, but it looks like that shipped around 1991?

https://bytepointer.com/masm/index.htm

Also some of the source files directly mention MASM, search this file:
https://github.com/lanceewing/agi/blob/main/src/SCROUT.ASM

[lance.ewing]

This page suggests that those directives were added in MASM 6.0, but it looks like that shipped around 1991?

Yeah, I noticed the same thing, e.g. that .if is in the there, but it ends with .endif rather than .end. I haven't tried MASM 6 yet, mainly for the reason that you mentioned, being that it wasn't around in 1987. Even MASM 5 was very new at the time that these .ASM files are from. I think I saw that it came out in Aug 1987. My assumption was that MASM 4 would have been used for most of the AGI source development.

Also some of the source files directly mention MASM, search this file:
https://github.com/lanceewing/agi/blob/main/src/SCROUT.ASM

Nice spotting! I hadn't noticed that yet. So that answers the question, i.e. it is definitely MASM, but for whatever reason, it doesn't like the code. I'm running it without any command line options though, other than the name of the source file. Not sure if I need to enable a feature somehow. Or maybe it gets pre-processed by something first. If I try to use the MWC (Mark Williams compiler) with the .ASM files, they delegate to MASM, but then it gets the same syntax errors. Something is missing.

[mnicolella]

I think it's also possible that Sierra got preview builds of MASM from Microsoft, perhaps that included the .if and other directive features ahead of them shipping in an official release. I wonder if you hunt through the MASM 4.x or 5.x executables if you would find any reference to these things, maybe behind an undocumented flag?

Here's something else that's an interesting point... the DOS 4.0 source code is on github, and includes usages of these directives -- and DOS4 shipped around 86-88?
https://github.com/microsoft/MS-DOS/blob/2d04cacc5322951f187bb17e017c12920ac8ebe2/v4.0/src/SELECT/S_DISPLY.ASM#L64

[mnicolella]

Oh, here's the file from the DOS repo that contains the macro definitions for .if and .endif

https://github.com/microsoft/MS-DOS/blob/2d04cacc5322951f187bb17e017c12920ac8ebe2/v4.0/src/INC/STRUC.INC#L320

Perhaps this file or something like it was distributed with earlier MASM versions? Or maybe Sierra had their own, or was given it from Microsoft?

[lance.ewing]

Yeah, it is starting to look like these were defined as macros, given that that example you found has shown that it is possible to define very similar macros. I think we can deduce what the macros contained based on comparing the usage of these macros with a disassembly of the AGI interpreter.