Fragments of the original AGI specs and interpreter code

[lance.ewing]

Yeah, I can confirm that what you've shown is definitely a BRIEF macro. I've found two snippets like that on other disks. The first (on SQ3) was from a fragment of a file called BUFFERS.M and the fragment was as follows:

Code: [Select]

ark (atoi (substr button_text (+ (rindex button_text ";") 1))))
(_dialog_esc)
(returns TRUE)
)
)
)
)

You can see the full version of what this came from at the following location (someone seems to have put a circa 1988 install of BRIEF on this site):

http://koti.mbnet.fi/juhal87/juhallebackupit/taimiDOSclevy/BR/MACROS/BUFFERS.M

The other fragment was from Black Cauldron and appears as follows:

Code: [Select]

change_window DOWN)
(top_of_buffer)
(search_fwd ":exit" 0)
(move_rel 5 0)
(refresh)
)
)

;else
(= addStr useStr)
))))

(= addStr (substr addStr 1 (- (strlen addStr) 1)))
(= addStr (+ "\t" (+ addStr "\n")))

(returns addStr)
)
)

This doesn't appear to match any of the standard macros that come with BRIEF, at least from what I've been able to tell. So I'm wondering if it is one of their custom macros.

On the subject of custom BRIEF macros, I found this fragment of a directory cluster on KQ1SCI that appears to show the names of some of their custom BRIEF macros. I'm basing this on the fact that the filenames contain SCI and that the extensions of the files are .M and .CM, which are the source and compiled BRIEF macro extensions.

Code: [Select]

Cluster# 694: EOF, appears after the end of: INSTGAME.BAT
0200                       20 43 4D 20 20 00 00 00 00          CM  .... 
0210  00 00 00 00 00 00 25 78 86 14 5A 0F B1 00 00 00  ......%x..Z.....  :Exit.CM             177 06-Apr-1990 15:01:10 [cluster=3930]
0220  53 43 20 20 20 20 20 20 4D 20 20 20 00 00 00 00  SC      M   .... 
0230  00 00 00 00 00 00 68 77 86 14 59 0F 53 02 00 00  ......hw..Y.S...  SC.M                 595 06-Apr-1990 14:59:16 [cluster=3929]
0240  53 43 45 52 52 20 20 20 43 4D 20 20 00 00 00 00  SCERR   CM  .... 
0250  00 00 00 00 00 00 B1 99 58 13 D9 0E 0B 07 00 00  ........X.......  SCERR.CM            1803 24-Oct-1989 19:13:34 [cluster=3801]
0260  53 43 45 52 52 20 20 20 4D 20 20 20 00 00 00 00  SCERR   M   .... 
0270  00 00 00 00 00 00 F0 60 57 13 D6 0E 17 10 00 00  .......`W.......  SCERR.M             4119 23-Oct-1989 12:07:32 [cluster=3798]
0280  53 43 49 20 20 20 20 20 43 4D 20 20 00 00 00 00  SCI     CM  .... 
0290  00 00 00 00 00 00 22 6D 09 13 4B 0E 1D 06 00 00  ......"m..K.....  SCI.CM              1565 09-Aug-1989 13:41:04 [cluster=3659]
02A0  53 43 49 20 20 20 20 20 4D 20 20 20 00 00 00 00  SCI     M   .... 
02B0  00 00 00 00 00 00 C8 79 8E 12 FB 0D 57 14 00 00  .......y....W...  SCI.M               5207 14-Apr-1989 15:14:16 [cluster=3579]
02C0  53 43 49 4B 45 59 53 20 43 4D 20 20 00 00 00 00  SCIKEYS CM  .... 
02D0  00 00 00 00 00 00 76 44 68 13 E4 0E D0 09 00 00  ......vDh.......  SCIKEYS.CM          2512 08-Nov-1989 08:35:44 [cluster=3812]
02E0  53 43 49 4B 45 59 53 20 4D 20 20 20 00 00 00 00  SCIKEYS M   .... 
02F0  00 00 00 00 00 00 6F 44 68 13 E2 0E C7 0C 00 00  ......oDh.......  SCIKEYS.M           3271 08-Nov-1989 08:35:30 [cluster=3810]
0300  53 45 41 52 43 48 20 20 43 4D 20 20 00 00 00 00  SEARCH  CM  .... 
0310  00 00 00 00 00 00 EA 51 0F 11 E7 0A 8C 18 00 00  .......Q........  SEARCH.CM           6284 15-Aug-1988 10:15:20 [cluster=2791]
0320  53 45 41 52 43 48 20 20 4D 20 20 20 00 00 00 00  SEARCH  M   .... 
0330  00 00 00 00 00 00 D7 66 8E 11 69 0D 13 42 00 00  .......f..i..B..  SEARCH.M           16915 14-Dec-1988 12:54:46 [cluster=3433]
0340  53 50 20 20 20 20 20 20 43 4D 20 20 00 00 00 00  SP      CM  .... 
0350  00 00 00 00 00 00 A9 72 09 13 51 0E DE 0B 00 00  .......r..Q.....  SP.CM               3038 09-Aug-1989 14:21:18 [cluster=3665]
0360  53 50 20 20 20 20 20 20 4D 20 20 20 00 00 00 00  SP      M   .... 
0370  00 00 00 00 00 00 84 74 09 13 53 0E 98 13 00 00  .......t..S.....  SP.M                5016 09-Aug-1989 14:36:08 [cluster=3667]
0380  53 54 41 52 54 55 50 20 43 4D 20 20 00 00 00 00  STARTUP CM  .... 
0390  00 00 00 00 00 00 F4 76 02 15 C8 10 2C 38 00 00  .......v....,8..  STARTUP.CM         14380 02-Aug-1990 14:55:40 [cluster=4296]
03A0  53 54 41 52 54 55 50 20 4D 20 20 20 00 00 00 00  STARTUP M   .... 
03B0  00 00 00 00 00 00 CA 52 4A 12 E4 0D 3E 1A 00 00  .......RJ...>...  STARTUP.M           6718 10-Feb-1989 10:22:20 [cluster=3556]

I mentioned the BRIEF macro snippets to the ex-Sierra employee that I was talking to earlier in the week and he said that he wasn't surprised. His exact words were that it was "the editor of choice of ALL programmers at Sierra" (at least until Windows came along).

Looking at the LISP-like / C-like syntax of the BRIEF macro language, I can't help but wonder how much it influenced the development of the SCI language. Based on the discovery of the Black Cauldron BRIEF macro snippet, they were obviously using BRIEF in the AGI days as well. So I suppose they were quite familiar with this kind of LISP/C hybrid syntax.

I'll take a look at your attachment now. I'm really hoping at some point we'll find some genuine SCI source on one of these game disks. If there are .BAT file fragments and BRIEF macro fragments, then it seems to be possible that we could. Hopefully luck will go our way. The closest I've found so far is a directory cluster fragment with names of SCI source files (from KQ1SCI again):

Code: [Select]

02C0  6B 71 31 0D 0A 20 20 20 53 43 20 20 00 00 00 00  kq1..   SC  .... 
02D0  00 00 00 00 00 00 DD 6C 2C 15 B2 16 3D 37 00 00  .......l,...=7..  kq1.SC             14141 12-Sep-1990 13:38:58 [cluster=5810]
02E0  52 4D 32 37 30 20 20 20 53 43 20 20 00 00 00 00  RM270   SC  .... 
02F0  00 00 00 00 00 00 34 4B 43 15 63 1F E9 5F 00 00  ......4KC.c.._..  RM270.SC           24553 03-Oct-1990 09:25:40 [cluster=8035]
0300  52 4D 33 36 30 20 20 20 53 43 20 20 00 00 00 00  RM360   SC  .... 
0310  00 00 00 00 00 00 48 6D 2C 15 67 16 0E 81 00 00  ......Hm,.g.....  RM360.SC           33038 12-Sep-1990 13:42:16 [cluster=5735]
0320  52 4D 34 35 30 20 20 20 53 43 20 20 00 00 00 00  RM450   SC  .... 
0330  00 00 00 00 00 00 B1 49 39 15 DC 14 8C 41 00 00  .......I9....A..  RM450.SC           16780 25-Sep-1990 09:13:34 [cluster=5340]
0340  52 4D 38 39 38 20 20 20 53 43 20 20 00 00 00 00  RM898   SC  .... 
0350  00 00 00 00 00 00 62 4D 2C 15 07 16 BC 0A 00 00  ......bM,.......  RM898.SC            2748 12-Sep-1990 09:43:04 [cluster=5639]
0360  52 4D 32 36 30 20 20 20 53 43 20 20 00 00 00 00  RM260   SC  .... 
0370  00 00 00 00 00 00 57 76 27 15 32 16 E5 C5 00 00  ......Wv'.2.....  RM260.SC           50661 07-Sep-1990 14:50:46 [cluster=5682]
0380  52 4D 33 34 30 20 20 20 53 43 20 20 00 00 00 00  RM340   SC  .... 
0390  00 00 00 00 00 00 84 76 27 15 4D 16 82 08 00 00  .......v'.M.....  RM340.SC            2178 07-Sep-1990 14:52:08 [cluster=5709]
03A0  52 4D 33 37 30 20 20 20 53 43 20 20 00 00 00 00  RM370   SC  .... 
03B0  00 00 00 00 00 00 6B 78 27 15 E7 16 D2 3F 00 00  ......kx'....?..  RM370.SC           16338 07-Sep-1990 15:03:22 [cluster=5863]
03C0  52 4D 32 35 30 20 20 20 53 43 20 20 00 00 00 00  RM250   SC  .... 
03D0  00 00 00 00 00 00 9D 92 41 15 72 1F 49 A0 00 00  ........A.r.I...  RM250.SC           41033 01-Oct-1990 18:20:58 [cluster=8050]
03E0  52 4D 32 30 30 20 20 20 53 43 20 20 00 00 00 00  RM200   SC  .... 
03F0  00 00 00 00 00 00 C8 7A 41 15 FF 06 42 6A 00 00  .......zA...Bj..  RM200.SC           27202 01-Oct-1990 15:22:16 [cluster=1791]


[lance.ewing]

It seems I've overlooked the most interesting file of the bunch I had saved before. I don't think I've looked past all the pages of the memory map (first part of the file) before, because otherwise I'd probably remembered, as it's pretty spectacular: C code of the interpreter complete with comments, programmer initials and everything.

Wow! Yeah, this is amazing. Can you remember what game disk it game from? I'd be keen to locate it myself to see how this appeared on the disk.

Also there's part of a batch file in there, creating a playable copy of the game(?):

Code: [Select]

copy agi.exe %2
copy *.ovl %2
setstr gameID %1 %2agi.exe
copy ..\src\agi.doc %2


Very interesting. I'm not sure that this was for creating a releasable version of the game though, since that agi.doc file is presumably the AGI specs that you found fragments of earlier. But given that you did find fragments of the AGI specs on an original game disk then maybe these commands were run on the original disk that you found the spec fragments on. But then again, given that this snippet of a batch file existed on a disk, it might be related to the dirty buffer issue I've been proposing. If a batch file containing these commands was run maybe the same day as a game disk was built, then both this fragment of a batch file, and the fragments of the AGI specs, may still have been in a buffer somewhere.

Not sure about the "over copy" bug. It might be a combination of both (buffer + deleted data), since the attached 210kb of data seems a bit too much to be buffered.

Yeah, I agree with this. That interpreter code must have been from a deleted file. It's way too big, and also relatively unfragmented. It suggests that there was a run off clusters on the original disk that were marked as free but that contained that interpreter code. That would have been from a deleted file. Most of the disks I've looked at so far have very few clusters marked as "free" with anything interesting on them though. They're mostly fill of the 0xF6 format byte. There are a few exceptions to this, but those cases have small bits of non-text binary content that isn't immediately recognisable. They must have come from a deleted file also. The dirty buffer "feature" must account for a lot of the data at the end of "end of file" clusters, particularly where the very next cluster is cleanly formatted.

Apparently the "duplication department" at Sierra did re-use disks, particularly when a run of copies of a game had been made for which they found a bug shortly afterwards. Those would be put aside and reused, I assume with the Formasters again for the next run.

Just copying with (X)COPY would have taken the same amount of time as doing a full DISKCOPY then, and I suppose that's something that would've been noticed at one point.

For full disk copying, they used the Formaster devices. They were apparently similar to the KryoFlux in that they didn't care about the format of the disks they were copying. They would apparently simply copy the raw data, and from a few things I've read online, it seemed that such machines could copy whole floppies in as little as 20 seconds.

For preparing the master set of disks for loading in to the Formaster, it seems that this must have been a batch file that simply copied the files one by one to the floppy disks. This is based on the batch file fragment for LSL2 that was discussed earlier in this thread.

I'm familiar with copies of the FAT being shattered all over the disk, but I don't know what causes it. I remember this one time, one of my disks somehow failed (corrupt filenames, file sizes of gigabytes). I inspected the disk with a disk editor and saw multiple copies of the FAT (or at least part of it) of one of my hard disk partitions there, which really struck me as odd. Again, no idea what causes it, but you'll see it on a lot of disks.

One thing I know for certain is that floppies had at least two copies of the FAT on them, one after the other. This was in case one of them became faulty.

[lskovlun]

Yeah, I agree with this. That interpreter code must have been from a deleted file.

A backup disk? Some backup tools of the era produced a single big file that spanned a disk (or several disks).

[lance.ewing]

Yeah, perhaps. Did such backup programs do any compression? I have noticed EXE file names in some of this junk data suggesting that a tool called Fast Back might have been on their hard disks.

[lance.ewing]

Here are some of the directory entry cluster fragments that impy that FastBack was on their hard disks. Firstly from LSL2 disk 1 (720K version):

Code: [Select]

Cluster# 274: EOF, appears after the end of: LSL2.QA
00B0                                               00                 . 
00C0  44 56 20 20 20 20 20 20 20 20 20 10 00 00 00 00  DV         ..... 
00D0  00 00 00 00 00 00 78 8E E1 10 6E 00 00 00 00 00  ......x...n.....  DV            <DIR>      01-Jul-1988 17:51:48 [cluster=110]
00E0  45 50 53 4F 4E 20 20 20 20 20 20 10 00 00 00 00  EPSON      ..... 
00F0  00 00 00 00 00 00 DA A2 33 10 7A 00 00 00 00 00  ........3.z.....  EPSON         <DIR>      19-Jan-1988 20:22:52 [cluster=122]
0100  46 41 53 54 42 41 43 4B 20 20 20 10 00 00 00 00  FASTBACK   ..... 
0110  00 00 00 00 00 00 6F 84 33 10 79 00 00 00 00 00  ......o.3.y.....  FASTBACK      <DIR>      19-Jan-1988 16:35:30 [cluster=121]
0120  4B 51 34 20 20 20 20 20 20 20 20 10 00 00 00 00  KQ4        ..... 
0130  00 00 00 00 00 00 F5 71 27 11 79 01 00 00 00 00  .......q'.y.....  KQ4           <DIR>      07-Sep-1988 14:15:42 [cluster=377]
0140  4C 4C 32 20 20 20 20 20 20 20 20 10 00 00 00 00  LL2        ..... 
0150  00 00 00 00 00 00 B8 3D 51 10 41 00 00 00 00 00  .......=Q.A.....  LL2           <DIR>      17-Feb-1988 07:45:48 [cluster=65]
0160  4D 4F 55 53 45 20 20 20 20 20 20 10 00 00 00 00  MOUSE      ..... 
0170  00 00 00 00 00 00 F1 BA 33 10 63 00 00 00 00 00  ........3.c.....  MOUSE         <DIR>      19-Jan-1988 23:23:34 [cluster=99]
0180  53 59 53 54 45 4D 20 20 20 20 20 10 00 00 00 00  SYSTEM     ..... 
0190  00 00 00 00 00 00 A4 5B 34 11 93 22 00 00 00 00  .......[4.."....  SYSTEM        <DIR>      20-Sep-1988 11:29:08 [cluster=8851]
01A0  53 59 53 54 45 4D 20 20 42 41 4B 10 00 00 00 00  SYSTEM  BAK..... 
01B0  00 00 00 00 00 00 EF 43 C8 10 4E 00 00 00 00 00  .......C..N.....  SYSTEM.BAK    <DIR>      08-Jun-1988 08:31:30 [cluster=78]
01C0  54 45 4C 45 43 4F 4D 4D 20 20 20 10 00 00 00 00  TELECOMM   ..... 
01D0  00 00 00 00 00 00 D7 A4 33 10 6C 00 00 00 00 00  ........3.l.....  TELECOMM      <DIR>      19-Jan-1988 20:38:46 [cluster=108]
01E0  54 4F 4F 4C 53 20 20 20 20 20 20 10 00 00 00 00  TOOLS      ..... 
01F0  00 00 00 00 00 00 0F A5 33 10 4A 00 00 00 00 00  ........3.J.....  TOOLS         <DIR>      19-Jan-1988 20:40:30 [cluster=74]

The assumption here is that a directory containing such sub-directories is unlikely to have been on a floppy (both KQ4 and LSL2 on the same floppy? And FastBack as well?). And besides, the SYSTEM directory starts at a cluster that is way too big to be on a floppy disk.

And this one from KQ4 disk 1 (720K version):

Code: [Select]

Cluster# 575: EOF, appears after the end of: CGA320C.DRV
0140                          43 4F 4D 20 00 00 00 00          COM .... 
0150  00 00 00 00 00 00 14 7E 49 10 50 01 B8 A1 00 00  .......~I.P.....  ??????.?.COM       41400 09-Feb-1988 15:48:40 [cluster=336]
0160  50 54 20 20 20 20 20 20 4C 41 53 20 00 00 00 00  PT      LAS .... 
0170  00 00 00 00 00 00 9A 8C 8D 11 1F 02 04 00 00 00  ................  PT.LAS                 4 13-Dec-1988 17:36:52 [cluster=543]
0180  46 42 20 20 20 20 20 20 45 58 45 20 00 00 00 00  FB      EXE .... 
0190  00 00 00 00 00 00 73 AD 44 10 66 01 0C 02 03 00  ......s.D.f.....  FB.EXE            197132 04-Feb-1988 21:43:38 [cluster=358]
01A0  53 54 41 52 54 55 50 20 46 42 20 20 00 00 00 00  STARTUP FB  .... 
01B0  00 00 00 00 00 00 DD 80 02 11 20 02 81 02 00 00  .......... .....  STARTUP.FB           641 02-Aug-1988 16:06:58 [cluster=544]
01C0  43 4F 4E 46 33 38 34 20 53 59 53 20 00 00 00 00  CONF384 SYS .... 
01D0  00 00 00 00 00 00 62 81 F3 10 22 02 28 00 00 00  ......b...".(...  CONF384.SYS           40 19-Jul-1988 16:11:04 [cluster=546]
01E0  57 48 41 54 20 20 20 20 45 58 45 20 00 00 00 00  WHAT    EXE .... 
01F0  00 00 00 00 00 00 00 00 FF 0E 23 02 62 0C 00 00  ..........#.b...  WHAT.EXE            3170 31-Jul-1987 00:00:00 [cluster=547]

Both FB.EXE and STARTUP.FB were apparently part of FastBack:

https://repositories.lib.utexas.edu/bitstream/handle/2152/12969/SNES_audio_disk_FILE.txt?sequence=6

There's a wikipedia page on it here:

https://en.wikipedia.org/wiki/FastBack

[lskovlun]

Replying here as I don't have a SHP account:

It is interesting to note in the KQ1 directory listing that Andrew posted on SHP that the room numbers implied by that listing are not the room numbers used in the final game. The final game has rooms 0 to 95 and a bunch of scripts in the high range.

[lance.ewing]

The starting cluster numbers for those are another example where they can't actually have ever been on the floppy disk. So that must be a directory that was on the hard disk from which the game files were copied. It could be for a different game. We might be able to work out what game it was. The kq1.SC file is a strong clue that it was KQ1 though. Is it possible that different releases of the game used different script numbers? It is possible (but seems unlikely) that the number specified in the .SC file for the script is different than what the filename implies.

[lance.ewing]

I've been running searches on Google for a few weeks now, trying to find something that explains the phenomenon I've been seeing in the unused parts of end-of-file clusters. It helps to know the proper terminology, and last night I finally found a document that mentioned the phrase "slack space". After doing a few searches with this, I started finding a lot more discussions about this part of a partially used sector. It is sometimes also referred to as "file slack".

My theory previously was that DOS was doing some sort of "over copying", filling the unused part of the cluster with whatever happened to be in its buffer beyond the end of the data that directly related to the file it was copying. About a week ago I started wondering instead whether DOS was copying the file from the hard-disk to the floppy disk in whole sectors, so for the last sector used by a file, I was starting to wonder whether it was copying the whole of the sector from the HD rather than only to the end of the file data. It occurred to me that maybe this was easier for it, and probably faster.

But if this were the case, then surely there must be references to something like this on the Internet.

Well this morning I found the following document that appears to describe behaviour similar to my first theory:

http://www.drtomoconnor.com/3100/3100lect08a.htm

Here's a copy-and-paste of the interesting paragraph:

Clusters vary in length depending on the operating system and size of the partition. Larger cluster sizes have more forensic value. With normal 512 byte sectors, if there is not enough data in the file to fill the last sector in the cluster, DOS/Windows makes up the difference by padding the remaining space with data from the memory buffers.  When this happens (often with a rather unsophisticated criminal or on an older machine), the file slack to look for is something called RAM slack that potentially contains information from previous work sessions as well as whole, complete sentences in their original wording. If the computer has been left on for several days, the file slack areas will contain a tremendous amount of information. RAM slack technically refers to the last sector of a file, but that is only because of the way borrowing from memory works in this case.  When a computer realizes it doesn't have enough RAM, it draws upon additional sectors to round out the block size for the last cluster, and then, another type of slack is created -- drive slack.  Unlike RAM slack which comes from memory, drive slack may very well contain what was stored on the computer long before, not just from previous work sessions.  If you are looking for things on a computer from a previous owner, drive slack is where to look.

Maybe the first theory was closer to the mark then but it doesn't seem to explain some of what I've been seeing. I'll lay out some examples when I get a chance.

Here's another web page discussing the terminology:

http://niiconsulting.com/checkmate/2006/06/file-slack-vs-ram-slack-vs-drive-slack/

[lance.ewing]

I just read another page claiming that the slack between the end of a file and the end of the sector is usually RAM slack, and the slack beyond that to the end of the cluster is "drive slack". The latter would be whatever was already on the disk and the former would be from RAM at the time of the copy.

[lance.ewing]

There's a good description of RAM slack on pages 752-753 of the following book:

https://books.google.co.uk/books?id=t0J2QcUtMKcC&pg=PA752#v=onepage&q&f=false

RAM slack is created from the buffers on Windows and DOS-based systems. Buffers can be thought of as plumbing used by the operating system, and the number of buffers on a Windows/DOS-based system is set in the CONFIG.SYS file, for example, buffers=30. The buffers reside in the random access memory (RAM) of the computer system, and the contents of the buffers can potentially contain any data or data fragments created, viewed, or printed during a computer work session. RAM slack is written to the first sector of the last cluster of the file. In Microsoft-based computers, sectors are small storage blocks that hold 512 bytes of data. Clusters are made up of varying numbers of sectors, depending of the size of the storage device and the operating system involved. RAM slack will always be in the first sector of the last cluster of the file, but RAM slack cannot contain more than 512 bytes of data. RAM slack is only a concern on Windows, Windows 95, Windows 98, and DOS-based systems because Dinows NT, Windows 2000 and Windows XP-based computer systems automatically scrub all relevant data contained in RAM slack.

And another good description here to complement the above, which gets even closer to describing my initial thoughts on what might be causing this:

https://books.google.co.uk/books?id=J8h8VWUmDuYC&pg=PA63#v=onepage&q&f=false

While all modern operating systems pad the written sector with null bytes, this was not always the case. MS-DOS and older DOS-based version of Microsoft Windows would pad the rest of the sector out with whatever contents of memory happened to be next to data being written. These data, between the end of allocated data and the beginning of previously allocted data, became known as RAM slack. Given this, RAM slack could potentially contain data that were never written to the disk, such as cryptographic keys or passphrases.

[lance.ewing]

The following Dr Dobbs article mentions something very interesting:

http://www.drdobbs.com/beware-of-file-slack-attacks/184405835

Flaws in software that copies files has been known to read beyond the logical end of a file and up to the physical boundary of the final cluster by mistake, transferring file slack from one place to another when the slack should be ignored by software that manipulates logical files. Whenever slack is copied as part of a file copy, an unknown quantity of potentially sensitive data is at risk of theft. For this reason, tools exist that will wipe file slack automatically as files are created or copied.

The above description matches my second theory on how this data may have got in the end of the sector, which was that the DOS copy tool (or a particular version over it) might have been copying whole sectors from the HD to the floppy disk, including the final sector being copied in whole.

It is possible that the original Sierra game disks suffered from three different issues (all of which must have been common in the 80s):

• Deleted Files: Data present on the floppy disk already, from deleted files. I think this would be rare to find on an original Sierra game disk (since they probably used new disks a lot of the time), but possibly the AGI specs fragments example, and clearly the AGI interpreter code mentioned earlier in this thread, were examples of this type.
• RAM Slack: When the last sector used by a file is padded out with data that is in the buffer from earlier I/O activity. This must have been read in to the buffer at some point after the PC was most recently turned on. I guess we don't know how often they turned off their PCs in the Sierra office.
• Flaws in file copy tools: Where a file copy tool reads beyond the end of a file, up to the end of the sector or cluster, thereby copying over the slack with the file. It is possible that the additional slack that such a tool is copying with the file was originally the result of RAM slack or deleted files at some point in the past, potentially months or even years in the past.

The following is a report of a very similar observation to what we're seeing with the Sierra disks, except that in the example below, it is on the Microsoft C 5.1 disks from 1988, and the fragments are from emails. It is from an OS/2 site, but the author points out that it isn't clear what host OS was used to created the disks:

http://www.os2museum.com/wp/careful-with-that-buffer/

[lance.ewing]

As mentioned in the "Original SCI syntax" thread, disks of AGI releases sometimes contain remnants of internal data.

Below are some fragments of the original AGI specs. Missing parts are marked as "@@@".

I have now verified that the AGI spec fragments came from the original LSL1, disk 1. The exact fragments that HWM posted appear in that order on the disk. The interesting bit is that they all appear in the second half of a cluster. For 720K disks, the cluster size is 2 sectors, i.e. 1024 bytes. So in every case where the AGI spec fragments appear, the cluster offset of the start of the AGI spec fragment is 0x200 (512). And in the cases where the "slack space" starts before the beginning of the second sector in the cluster, the slack in the first sector is always unrelated to the AGI fragment that appears in the second sector of the cluster, for example:

Code: [Select]

Cluster# 81: EOF, appears after the end of: _LL.BAT
0020                 20 20 20 20 20 20 10 00 00 00 00             ..... 
0030  00 00 00 00 00 00 0F 50 EA 0E 80 19 00 00 00 00  .......P........ 
0040  47 52 41 46 20 20 20 20 4F 20 20 00 00 00 00 00  GRAF    O  ..... 
0050  00 00 00 00 00 00 43 45 9C 0E 95 19 1C 03 00 00  ......CE........ 
0060  43 50 43 20 20 20 20 20 4F 20 20 00 00 00 00 00  CPC     O  ..... 
0070  00 00 00 00 00 00 F7 72 D9 0E 96 19 02 02 00 00  .......r........ 
0080  52 45 4C 4F 43 20 20 20 4F 20 20 00 00 00 00 00  RELOC   O  ..... 
0090  00 00 00 00 00 00 22 71 0E 0D 97 19 BA 01 00 00  ......"q........ 
00A0  53 54 41 54 55 53 20 20 4F 20 20 00 00 00 00 00  STATUS  O  ..... 
00B0  00 00 00 00 00 00 F2 84 39 0D 98 19 50 02 00 00  ........9...P... 
00C0  4C 4F 41 44 50 52 4F 47 4F 20 20 00 00 00 00 00  LOADPROGO  ..... 
00D0  00 00 00 00 00 00 E5 85 33 0B 99 19 F0 03 00 00  ........3....... 
00E0  44 49 52 20 20 20 20 20 4F 20 20 00 00 00 00 00  DIR     O  ..... 
00F0  00 00 00 00 00 00 E6 85 33 0B 9A 19 CE 01 00 00  ........3....... 
0100  57 52 54 42 4F 4F 54 20 4F 20 20 00 00 00 00 00  WRTBOOT O  ..... 
0110  00 00 00 00 00 00 E8 85 33 0B 9B 19 0D 04 00 00  ........3....... 
0120  44 4F 53 4C 4F 41 44 20 4F 20 20 00 00 00 00 00  DOSLOAD O  ..... 
0130  00 00 00 00 00 00 FC 40 DE 0E 9C 19 10 04 00 00  .......@........ 
0140  42 4F 4F 54 20 20 20 20 53 20 20 00 00 00 00 00  BOOT    S  ..... 
0150  00 00 00 00 00 00 19 52 3E 0D 9D 19 96 1D 00 00  .......R>....... 
0160  43 4F 4C 4F 52 20 20 20 53 20 20 00 00 00 00 00  COLOR   S  ..... 
0170  00 00 00 00 00 00 D0 85 33 0B A6 19 BE 01 00 00  ........3....... 
0180  52 45 41 44 44 49 53 4B 4F 20 20 00 00 00 00 00  READDISKO  ..... 
0190  00 00 00 00 00 00 EC 85 33 0B A9 19 0B 01 00 00  ........3....... 
01A0  4D 45 53 53 41 47 45 20 53 20 20 00 00 00 00 00  MESSAGE S  ..... 
01B0  00 00 00 00 00 00 21 71 90 0E AA 19 83 10 00 00  ......!q........ 
01C0  4C 4F 41 44 20 20 20 20 53 20 20 00 00 00 00 00  LOAD    S  ..... 
01D0  00 00 00 00 00 00 3C 52 3E 0D AD 19 34 17 00 00  ......<R>...4... 
01E0  4C 4F 41 44 50 52 4F 47 53 20 20 00 00 00 00 00  LOADPROGS  ..... 
01F0  00 00 00 00 00 00 D7 85 33 0B B1 19 AB 14 00 00  ........3....... 
0200  73 20 69 73 20 64 6F 6E 65 20 62 79 0D 0A 09 61  s is done by...a 
0210  6E 69 6D 61 74 65 2E 6F 62 6A 28 29 2E 0D 0A 0D  nimate.obj().... 
0220  0A 65 6E 64 2E 6F 66 2E 6C 6F 6F 70 28 4F 42 4A  .end.of.loop(OBJ 
0230  45 43 54 2C 20 46 4C 41 47 29 0D 0A 09 52 65 73  ECT, FLAG)...Res 
0240  65 74 20 74 68 65 20 66 6C 61 67 2E 20 20 49 6E  et the flag.  In 
0250  63 72 65 6D 65 6E 74 20 74 68 65 20 63 65 6C 09  crement the cel. 
0260  6E 75 6D 62 65 72 20 61 74 20 65 61 63 68 20 61  number at each a 
0270  6E 69 6D 61 74 69 6F 6E 20 63 79 63 6C 65 2E 0D  nimation cycle.. 
0280  0A 09 57 68 65 6E 20 74 68 65 20 6C 61 73 74 20  ..When the last   
0290  63 65 6C 20 6F 66 20 74 68 65 20 6C 6F 6F 70 20  cel of the loop   
02A0  69 73 20 72 65 61 63 68 65 64 2C 20 73 74 6F 70  is reached, stop 
02B0  20 63 79 63 6C 69 6E 67 20 61 6E 64 20 73 65 74   cycling and set 
02C0  20 74 68 65 20 66 6C 61 67 2E 0D 0A 0D 0A 72 65   the flag.....re 
02D0  76 65 72 73 65 2E 63 79 63 6C 65 28 4F 42 4A 45  verse.cycle(OBJE 
02E0  43 54 29 0D 0A 09 43 79 63 6C 65 20 74 68 65 20  CT)...Cycle the   
02F0  6F 62 6A 65 63 74 20 66 72 6F 6D 20 74 68 65 20  object from the   
0300  63 75 72 72 65 6E 74 20 63 65 6C 20 6E 75 6D 62  current cel numb 
0310  65 72 20 74 6F 20 63 65 6C 20 30 2E 20 20 57 68  er to cel 0.  Wh 
0320  65 6E 20 63 65 6C 20 30 0D 0A 09 69 73 20 72 65  en cel 0...is re 
0330  61 63 68 65 64 2C 20 73 74 61 72 74 20 61 67 61  ached, start aga 
0340  69 6E 20 61 74 20 74 68 65 20 6C 61 73 74 20 63  in at the last c 
0350  65 6C 20 6F 66 20 74 68 65 20 63 75 72 72 65 6E  el of the curren 
0360  74 20 6C 6F 6F 70 2E 0D 0A 09 0D 0A 72 65 76 65  t loop......reve 
0370  72 73 65 2E 6C 6F 6F 70 28 4F 42 4A 45 43 54 2C  rse.loop(OBJECT, 
0380  20 46 4C 41 47 29 0D 0A 09 52 65 73 65 74 20 74   FLAG)...Reset t 
0390  68 65 20 66 6C 61 67 2E 20 20 44 65 63 72 65 6D  he flag.  Decrem 
03A0  65 6E 74 20 74 68 65 20 63 65 6C 20 6E 75 6D 62  ent the cel numb 
03B0  65 72 20 61 74 20 65 61 63 68 20 61 6E 69 6D 61  er at each anima 
03C0  74 69 6F 6E 20 63 79 63 6C 65 2E 0D 0A 09 57 68  tion cycle....Wh 
03D0  65 6E 09 63 65 6C 20 30 20 69 73 20 72 65 61 63  en.cel 0 is reac 
03E0  68 65 64 2C 20 73 74 6F 70 20 63 79 63 6C 69 6E  hed, stop cyclin 
03F0  67 20 61 6E 64 20 73 65 74 20 74 68 65 20 66 6C  g and set the fl

These observations appear to confirm what was claimed regarding slack space, i.e. that the final sector used by a file is filled to the end of that sector with "ram slack", and any remaining sectors in the same cluster are filled with "drive slack" (i.e. deleted file content already on the disk). In the case of the second type of slack, I still have a feeling that it wasn't actually on the floppy disk, but was instead "drive slack" on the hard disk from which it was copied, and that a flaw in the copy tool copied the "file slack" (both "ram" and "drive" slack) along with the file, up to the end of the last cluster used by the file, on to the floppy.

One thing that I find very strange is that all the literature in the computer forensics domain claim that "ram slack" is the result of randomly selected RAM contents. It implies that DOS somehow randomly chose what part of RAM to "fill" the rest of the sector with. I've even found references that implied that DOS had to do this in order to function properly. I find both of those suggestions very difficult to believe. Most of us here are programmers, so we can imagine that randomly selecting memory contents to "pad" the rest of a sector with is actually an overhead to the file writing process rather than an unavoidable necessity. It seems far more likely that DOS's memory buffers dealt with sectors as a whole and that the final sector was simply written out as a sector-sized chunk from the memory buffer and that if the last part happened to have data from an earlier read file, then that would be written out with the final sector. It is possible that this additional "slack" was described as being random, simply because anything could be in the memory buffer from earlier reads, and that perhaps through assumptions made by people reading that, it became described as "randomly selected" memory contents. I actually find it quite worrying that the computer forensics literature on this is often coming out of the legal domain and what they are stating seems to be a misunderstanding of what is probably happening. I'm considering running a few tests using DOS 3.2 and DOS 3.3 on a couple of PC emulators to see if I can reproduce the generation of slack space, in an attempt to understand what was happening.

I was also planning to send a few questions Tim Patterson's way to see what his thoughts on this are. He wrote the original version of DOS. It may be that the whole slack space thing was introduced after his time, but I'm fairly sure it was there by DOS 3. All of the Sierra original disk examples I've seen are on floppy disks that were formatted with either version 3.2 or 3.3, and one that was formatted by Norton Utilities (which gives no indication about the DOS version).

What does all this have to do with AGI and SCI? It's the process of understanding how slack originated. I'm hoping that we might be able to find more original SCI source code on original game disks. We've already found fragments of BRIEF macros, which we know that they used. One of the macros even appears to be one of their custom BRIEF macros. So if this can appear in the slack space of a file's final cluster, and AGI specs, then it is certainly possible that SCI source code could appear in there.

As a final note, I read a few days ago on another site somebody asking the question of whether CDROMs from the DOS and earlier Windows days also suffered from slack space. It's quite possible I suppose. It is worth checking out.

[NewRisingSun]

I have sorted the various fragments of the source code found in the unused sectors of my Space Quest II Version 2.0D 720K into separate files based on what the first line comment said. It's not a complete interpreter source, obviously, but I found it enlightening that the PS/2 models 25 and 30 with their MCGA graphics adapter apparently caused Sierra much trouble (see EQUIPCHK.ASM).

[lance.ewing]

Hey thanks. That's great. - Is Space Quest II Version 2.0D 720K where the AGI interpreter source code fragments mentioned earlier originally came from? I remember asking the question of where it came from but can't remember if it was answered.

[lance.ewing]

I'm not sure if this has been discussed before (probably has been but I'm guessing in a long lost forum somewhere), but I've been wondering what version of the AGI interpreter the above source code fragments would have come from. NewRisingSun says above that the attached ZIP of source fragments came from SQ2 version 2.0D. According to HWM's interpreter list, that version of the game used interpreter version 2.936.

http://agisci.altervista.org/SVLIST094.TXT

 2.936 - King's Quest 3 2.14 [3/15/88] *
         Space Quest 2 2.0D *
         Space Quest 2 2.0F *

According to the AGI specs, version 2.936 had either 175 or 177 commands (different sites have a different number for this, but it doesn't really matter for this discussion). I don't think that the AGI source code fragments from the SQ2 v2.0D 720K disks is for version 2.936 of the interpreter though. Command 175 is discard.sound, but I can't find any evidence of that command in these source code fragments. There is a SOUND.C and a SOUND.H file, and inside SOUND.C there are functions to cover loading sounds, playing sounds, and stopping sounds, but nothing for discarding. I think it would be in that file if this were the 2.936 version. PushScript and PopScript exist though, so that means commands 171 and 172 are there. The source code for hold.key is not present but there is a HOLDKEY mentioned in the memory map, so I think we can say that command 173 was present. - I don't think command 174 is present though. Nothing related to it that I can find, in fact the priority base appears to be referenced as a constant in the code. Interpreter versions 2.915 and 2.917 (and possibly others around them) had 173 commands. I suppose that is the most likely version that the source relates to then.

Prior to the introduction of discard.sound, there were only discard.pic, discard.view, and discard.view.v. For both the pic and view discard commands, they have matching codes in the script event buffer that is used as part of the saved game restoration. A safe bet would be that discard.sound also had an action code in the script event buffer. A guess would be that it would be the next number after the code for overlay.pic. To be certain I guess I'll have to write a small test logic that uses discard.sound and see what appears in the saved game under 2.936.

Edit: I've just performed that check with a discard.sound running on 2.936. Nothing was added to the script events according to the saved game file that was produced. I can see the load.sound but not the discard.sound. I added both as part of the same logic compile.