Author Topic: Fragments of the original AGI specs and interpreter code  (Read 3706 times)

0 Members and 1 Guest are viewing this topic.

Offline HWM

Fragments of the original AGI specs and interpreter code
« on: May 30, 2015, 07:41:26 PM »
As mentioned in the "Original SCI syntax" thread, disks of AGI releases sometimes contain remnants of internal data.

Below are some fragments of the original AGI specs. Missing parts are marked as "@@@".

Code: [Select]

@@@

esult in the var.

div.v(VAR, VAR)
Parameters: destination, divisor
No alternate way to write it in the compiler.
Divide the first var by the second, leaving the quotient in the
first var.

lindirectv(VAR, VAR)
[Written as var1 @= var2]
Left indirect assignment of vars.  Takes the value in var1 and uses it
as the var number in which to store the value of var2.  In essence,
var1 is a 'pointer' to the var for the assignment.  Thus,

%var var1 39
%var var2 56
var1 = 27;

@@@

rks
on white and the picture being overlayed is probably not all white.
As in draw.pic(), a call to show.pic() is necessary to bring the
new picture to the foreground screen.

show.pri.screen()
Show the priority screen, then wait for ENTER or ESC before returning
to the game.  Generally implemented as a debug action.



------ View actions

load.view(VIEW)
Load the view whose number is given by the number.

load.view.f(var)
Load the view whose number is in the var.

discard.view(VI

@@@

 t.motion(), start.cycling(), normal.cycle(), observe.blocks(),
observe.horizon(), on.anything(), release.priority(), release.loop(),
and observe.objects() on the object and sets its direction to 0.

unanimate.all()
Unanimate all animated objects.  If you're going to draw a second picture
in a room (using draw.pic() without doing a new.room()) do this first
to make sure that you don't end up with a number of animated objects
which you don't want in the new picture.  You'll have to do an
anima

@@@

 T)
Erase the object from the screen.  Erased objects do not move or
interact with drawn objects.

position(OBJECT, NUM, NUM)
Parameters: object, x coord, y coord.
Set the position of the object (the lower left corner of its baseline)
to the (x, y) coords given in the numbers.  x is measured from the right
edge of the screen, y from the top.  If part of the object will be off
the screen, if the object is above the horizon and must observe it, or
if the object's baseline is on a priority lin

@@@

error to that corresponding to an object's
current direction.  A non-moving object (direction = 0) remains in
its last loop.

set.loop.f(OBJECT, VAR)
Set the object's loop to that in the var.

fix.loop(OBJECT)
Fix the object's loop.  It will no longer adjust to the direction
in which the object is moving.

release.loop(OBJECT)
Undo a fix.loop().  The object will now face in the direction
appropriate to its direction.

set.cel(OBJECT, NUM)
Set the object's cel to that in the number.  Cels                         

@@@

s is done by
animate.obj().

end.of.loop(OBJECT, FLAG)
Reset the flag.  Increment the cel number at each animation cycle.
When the last cel of the loop is reached, stop cycling and set the flag.

reverse.cycle(OBJECT)
Cycle the object from the current cel number to cel 0.  When cel 0
is reached, start again at the last cel of the current loop.

reverse.loop(OBJECT, FLAG)
Reset the flag.  Decrement the cel number at each animation cycle.
When cel 0 is reached, stop cycling and set the fl

@@@

nd background colors for text.  Not all combinations
will necessarily be supported on all machines.  We will try to
approximate where we can, but no guarantees.  The colors are:

0 black
1 dark blue
2 dark green
3 cyan
4 red
5 magenta
6 brown
7 light grey
8 dark grey
9 light blue
10 light green
11 light cyan
12 pink
13 light magenta
14 yellow
15 white

All combinations will (I believe) be supported on the Atari ST, Amiga,
and the NEC 9801.

On th

@@@

The number is the
number of figure eights to do.  This will be a no-op on some hardware,
so don't depend on it.



------ Screen handling

[* 25 lines total

configure.screen(NUM, NUM, NUM)
Parameters: picture row, input row, status row.
This call, which should be done as soon as the game starts, sets
where the various components of the screen are placed.  The first
number is the CHARACTER row number (starting at zero) for the upper
left corner of the picture, the second is the row n

@@@

ers: string number, message number.
Copy the message into the string given by the number.

get.string(NUM, MSGNUM, NUM, NUM)
Parameters: string number, message number, row, column.

Prints the message as a prompt at the given screen position, then
allows the user to enter the string for string number NUM.  If the
row is >24, the message will be printed at the current cursor
position.

Since string 0 is the prompt, set the prompt by
%message 1 "> "
set.string(0, 1);

The code to le

@@@

ilable to all said() tests for the remainder
of the current logic scan.

get.num(MSGNUM, VAR)
Prompt the user with the message and get a (purportedly) numeric reply.
Put the number into var.  If a non-numeric reply is typed, var will
be 0.



------ Input handling

prevent.input()
Clear the input line and do not accept input from the user.  Input
is off at startup.

accept.input()
Display the input line and accept input from the user.

set.key(NUM, NUM, NUM)
Parameters: low byte 

@@@

set.key(27, 0, c.pause); [ESC
set.key(18, 0, c.rgb.toggle); [^R
set.key(16, 0, c.new.prompt); [^P

set.key(joy.low.byte, button0, c.doit);
set.key(joy.low.byte, button1, c.useit);
set.key(joy.low.byte, button0.dbl, c.lookit);
set.key(joy.low.byte, button1.dbl, c.status);

The controller definitions are in 'sysdefs'.  You may map up to
29 keys.  More than one key may map to a single controller, but
a single key can't map to more than one controller.

****** Note that the key

@@@




Offline Collector

Re: Fragments of the original AGI specs and interpreter code
« Reply #1 on: May 30, 2015, 11:43:08 PM »
Were these separate files or in the VOL files?
KQII Remake Pic

Offline lance.ewing

Re: Fragments of the original AGI specs and interpreter code
« Reply #2 on: May 31, 2015, 03:07:02 AM »
The way in which it is fragmented, and the mention of "disks", suggests to me that these fragments were from disk sectors that were previously used by a file that had been deleted. When a file is deleted, it just marks the sectors as free in the FAT, i.e. it does't actually wipe the data on those sectors. So if their master copy of LSL had originally had the AGI specs on it, but was then deleted in prep for distribution, and if copies of the disk were made by doing a full disk copy (rather than individual files), then all copies would have this on them. You'd need to break out a disk sector hex viewer to see it though.

Something like that is my guess while we're waiting for HWM.

Offline HWM

Re: Fragments of the original AGI specs and interpreter code
« Reply #3 on: May 31, 2015, 07:34:35 AM »
Correct. Actually it's quite common for disks of older (1980's) PC releases to contain leftover data. I've encountered it a couple of times, although I only checked the AGI disks after it was discussed in the thread mentioned earlier. I checked some SCI disks too, but I don't think anything turned up there; maybe there were new mastering procedures in place by then.

The original thread mentioned another AGI release containing data as well, but I didn't have that one to check myself, so there's probably more out there.

EDIT: I've added the assembly code fragment as an attachment to this post. It consists of graphic subroutines, mainly for Hercules video. As opposed to the AGI specs, it's one big chunk, only the last part is missing some bits.
« Last Edit: May 31, 2015, 07:53:05 AM by HWM »

Offline lance.ewing

Re: Fragments of the original AGI specs and interpreter code
« Reply #4 on: May 31, 2015, 09:47:33 AM »
Unfortunately I don't have any of my original disks with me at the moment, otherwise I'd be quite eager to start looking over them. They're actually in storage on the other side of the world. I'm tempted to buy a few originals online again, just so I can do this.

You mentioned looking at SCI games. KQ4, LSL2, PQ2 and SQ3 are probably the best ones to take a look at, given they were the earliest. Perhaps they hadn't changed their process by that point. When thinking about where we might find original SCI source, I don't think this possibility has ever crossed my mind, i.e. look over the unused sectors on the original disks for data from deleted files. This might be a way of finding more snippets of real SCI source code. If for some reason in the creation of a particular game they wiped and reused a floppy to produce the packaged files for distribution, then it could have parts of anything still left on it. I think we'd need to focus on SCI games from the pre-CD days. My copy of QFG2 was still on floppies, so anything from 1988-1990 is probably best to take a look at the original disks of. It might be wishful thinking though. Perhaps the reason AGI specs and code was found is because its from earlier days before processes were tightened up.

As a question to everyone reading this, what original AGI and SCI games do we each have ready access to? I guess the biggest problem these days is having access to a floppy disk, and then crossing your fingers in hope that the disks aren't damaged. I've got a USB 3 1/2 inch floppy drive but nothing that can read the old 5 1/4 disks. Those old games usually came with both sets of disks though.

Offline Collector

Re: Fragments of the original AGI specs and interpreter code
« Reply #5 on: May 31, 2015, 12:52:04 PM »
Yes, I am aware of how files were deleted in DOS. I just would not have expected them to use a developmental disk for the master copy for production, so file fragments did not occur to me.

I have all of the AGI and SCI games, at least one version of each and several of some games. I have also started imaging my floppies with a KryoFlux, which would probably be able to capture file fragments, at least with a KryoFlux stream image. I recently was able to make a viable image of my KQ2 Tandy booter and converted the stream image into a Transcopy image. I have a special build of DOSBox that can boot from a Transcopy image. This handled the copy protection as if it was the original disk on my Win7 machine. http://www.kryoflux.com/
KQII Remake Pic

Offline lance.ewing

Re: Fragments of the original AGI specs and interpreter code
« Reply #6 on: May 31, 2015, 01:14:41 PM »
Yeah, as long as it is copying everything that is on the disk, which I'd assume that a disk image would do, then it would be perfect for the investigation. You could load those disk images in to a normal modern day hex editor and search for interesting text. If you had all of them as disk images, there might be a way to search across all of them in one go, maybe using a tool like strings or grep. You could start by seeing if you can find a match for some of the text that HWM posted. That would validate the approach and maybe confirm the game that it appeared in.

I also find it a little strange that the sold copies of games could contain copies of sectors for deleted files. Maybe it was a one-off based on the circumstances of how that particular game was prepared. Maybe it was a single dev working late trying to get things done and he was struggling to find a completely clean floppy. Or maybe reusing floppies was a common thing. I know I reused a lot.

Edit: Thinking about it a bit more, we could probably take each disk image and extract out only the unused sectors and the unused parts of partially used sectors of each image. That way we know instantly that whatever we find, it isn't part of the distributed game files. The images would contain the boot sector and FAT sectors as well. The bootable Tandy one is probably a different story though.
« Last Edit: May 31, 2015, 01:53:46 PM by lance.ewing »

Offline Collector

Re: Fragments of the original AGI specs and interpreter code
« Reply #7 on: May 31, 2015, 03:19:04 PM »
Let me know if there is one that you would like to explore. Keep in mind that a KryoFlux stream image can be several 10s of MB in size for a single floppy. You could almost just do an old MS-DOS undelete, if not too much of it has been overwritten. But then we are talking about the era when users may have not had hard drives and may have been writing save files to the disk. This is one thing that the KryoFlux controller program can do, is to identify altered sectors from the original creation of the disk.
KQII Remake Pic

Offline lance.ewing

Re: Fragments of the original AGI specs and interpreter code
« Reply #8 on: May 31, 2015, 03:39:12 PM »
You mention that it can identify altered sectors. Is it just that it knows the sector has been altered? Or can it help with working out what was written to that sector in the past? That sounds pretty incredible if it were possible. I'm guessing not though. But why is the KryoFlux stream so large if the floppy is only 360 /720K? I'm not familiar with it, so not sure what exactly it is reading. Is it not just data but the different levels of magnetism or something like that?

I guess that wouldn't be much help for the copies that were sold. We'd need the original master floppy to analyze its past. The copies would presumably be clean copies but if they were a full disk copy then they would contain copies of sectors for deleted files. - Yeah, I wonder what would happen if we tried using undelete on some of those disks?

What is the process of converting a KryoFlux stream image to an actual disk image with byte data in it?

Offline HWM

Re: Fragments of the original AGI specs and interpreter code
« Reply #9 on: May 31, 2015, 04:46:20 PM »
I just used a regular floppy drive and a tool called ImageDisk. This resulted in a binary copy of the full contents of the disk, so everything except for copy protection timing stuff / bad sectors, etc. Basically that's all you need in this case. I'm not that familiar with KryoFlux, but I'm pretty sure you'll just have to convert the KryoFlux stream image into a regular IMG file and after that you should be able to explore it (in readable format) using a hex editor. I assume you can use undelete, but I don't think it would work here, as the fragments probably don't have an entry in the FAT anymore.

I also find it a little strange that the sold copies of games could contain copies of sectors for deleted files. Maybe it was a one-off based on the circumstances of how that particular game was prepared. Maybe it was a single dev working late trying to get things done and he was struggling to find a completely clean floppy. Or maybe reusing floppies was a common thing. I know I reused a lot.

It really is quite common to see all kinds of unrelated data on officially sold copies, at least in PC games from the 80's. I backed up some of my oldest disks years ago and saw lots of removed content on retail disks, for instance there was this chess game that seemed to be mastered on a utility disk of a tape drive.

Offline Collector

Re: Fragments of the original AGI specs and interpreter code
« Reply #10 on: May 31, 2015, 05:40:47 PM »
The KryoFlux stream files are raw. It captures the entire disk, all sectors, formatting, hidden tracks, etc. There are a couple of emulators that can use the the streams, but they are mostly for archival purposes. DTC, the program that comes with the KryoFlux can also create standard images like IMG, ADF, IPF, CBM, etc., depending on the platform of the disk. One of the main advantages is that you can use it from any modern PC, Mac, Linux or Amiga via USB. If you are interested you can take a look at the manual http://www.kryoflux.com/download/kryoflux_manual.pdf

Here is the box I put together to read both 3.5" and 5.25" disks from various platforms. http://sierrahelp.com/forums/viewtopic.php?f=86&t=4213#p55211
KQII Remake Pic

Offline lance.ewing

Re: Fragments of the original AGI specs and interpreter code
« Reply #11 on: June 07, 2015, 06:39:40 PM »
It really is quite common to see all kinds of unrelated data on officially sold copies, at least in PC games from the 80's. I backed up some of my oldest disks years ago and saw lots of removed content on retail disks, for instance there was this chess game that seemed to be mastered on a utility disk of a tape drive.

To say I've been somewhat distracted for most of last week by the implications of the above posts is probably an understatement. I think there's a bit of a goldmine here. HWM is right about unrelated data regularly appearing on officially sold copies. This isn't something that happened here and there. It appears to have happened nearly all the time. Having looked at a few examples, I no longer think that this data was from a deleted file that was previously on the floppy disks. My hunch is that these floppies were as clean as they get prior to the game files being copied on to them. I'm now thinking that the DOS copy tool of that era had some kind of buffer "over copy" bug in it, where it might have had data in its buffer from earlier I/O activity and that when it reached the end of file, it continued writing to the end of the sector (or perhaps the end of the cluster) with whatever happened to be in its buffer. This is the kind of pattern that appears a lot on the original disks. There is usually some sort of junk data beyond the end of the file that is using that cluster, and its not random junk. Sometimes it contains bits of files that have no business being on the floppy, other times it has DOS directory-entry tables where the entries contain details for files that can't possibly have ever been on the floppy disk because their starting cluster numbers are way beyond the size of a floppy.

It's all hinting at some kind of "over copy" bug. If such a bug existed in DOS, we could find bits of just about anything on any original game disk. It could take years to fully investigate and document what we find. If such a bug existed, it would mean that the 360K vs 720K disk versions of a game have different junk on them, and it would mean that different numbered releases of a game have different junk on them. So for every floppy game they released, multiply that by the number of released versions of that game, then multiply that by 2 (for 360/720 disk versions), and then I guess we multiply that by the number of different languages each game was released in. And this assumes that for each version that only one set of master floppies was produced, from which all customer copies were made. It is possible instead that they produced multiple sets of masters for efficiency reasons. Basically what I'm suggesting is that on any occasion that they ran their script to copy the built game files to a floppy, there's a good chance that something else was copied along with it, perhaps whatever they happened to have run or were viewing earlier that day. Given that such junk appears after the end of a file, then disk 1 of most games would be the best place to look since it has a lot more files on it.

Any thoughts on whether such a bug existed in DOS back then? I've tried Googling for this but haven't turned up anything yet. Lot's of people mention that data from deleted files can show up in free and partially used clusters, but I've found no mention yet of DOS itself copying additional junk data after the end of a file. Feel free to completely blast this theory to pieces if you know that such a bug can't have existed.

Offline Collector

Re: Fragments of the original AGI specs and interpreter code
« Reply #12 on: June 08, 2015, 12:42:20 AM »
Some tool that they were using in place of XCOPY? When was XCOPY introduced?
KQII Remake Pic

Offline lance.ewing

Re: Fragments of the original AGI specs and interpreter code
« Reply #13 on: June 08, 2015, 05:29:23 PM »
The following "junk" data after the end of RESOURCE.006 on disk 6 of LSL2 720K version is the best clue of what DOS tool they used to prepare the master floppies:

Code: [Select]
Cluster# 339:
                              20 33 36 30 5C 72            360\r
65 73 6F 75 72 63 65 2E 30 30 35 20 20 20 20 61 esource.005    a
3A 0D 0A 63 6F 70 79 20 2A 2E 71 61 20 20 20 20 :..copy *.qa   
20 20 20 20 20 20 20 20 20 20 20 20 61 3A 0D 0A             a:..
76 6F 6C 61 62 65 6C 20 2F 6C 20 61 3A 4C 53 4C volabel /l a:LSL
32 20 44 69 73 6B 20 35 0D 0A 72 65 6D 20 07 0A 2 Disk 5..rem ..
0A 0A 0A 0A 20 20 20 20 20 20 20 20 20 20 20 20 ....           
20 20 20 20 20 20 20 43 68 61 6E 67 65 20 44 72        Change Dr
69 76 65 20 41 3A 20 74 6F 20 44 69 73 6B 20 23 ive A: to Disk #
36 2E 0D 0A 70 61 75 73 65 0D 0A 63 6F 70 79 20 6...pause..copy
33 36 30 5C 72 65 73 6F 75 72 63 65 2E 30 30 36 360\resource.006
20 20 20 20 61 3A 0D 0A 63 6F 70 79 20 2A 2E 71     a:..copy *.q
61 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 a               
20 61 3A 0D 0A 76 6F 6C 61 62 65 6C 20 2F 6C 20  a:..volabel /l
61 3A 4C 53 4C 32 20 44 69 73 6B 20 36 0D 0A 63 a:LSL2 Disk 6..c
64 20 20 20 2E 2E 0D 0A 72 65 6D 20 41 6C 6C 20 d   ....rem All
64 6F 6E 65 2E 07 1A 0D 0A 20 20 20 20 20 20 20 done.....       
20 20 20 20 20 20 20 62 3A 0D 0A 76 6F 6C 61 62        b:..volab
65 6C 20 2F 6C 20 62 3A 4C 53 4C 32 20 44 69 73 el /l b:LSL2 Dis

If I pull out only the ASCII text, including its formatting, then it looks like this:

Code: [Select]
360\resource.005    a:
copy *.qa                a:
volabel /l a:LSL2 Disk 5
rem 




                   Change Drive A: to Disk #6.
pause
copy 360\resource.006    a:
copy *.qa                a:
volabel /l a:LSL2 Disk 6
cd   ..
rem All done.
              b:
volabel /l b:LSL2 Dis

The first line is obviously truncated and would have started with "copy ", as in the example of disk 6.

What we appear to be seeing here is the batch file that was used to create the LSL2 master floppies. It appears to prompt the developer to change the disk prior to writing the files for a particular game disk to the new floppy. This is the opposite of what we see when installing a game on to the hard disk. This batch file is clearly writing the files from a hard disk on to empty floppies, and then it sets the label of each disk as it goes. These labels match the labels on the original disks.

So assuming this really was used to create the LSL2 master disks, then it suggests that the DOS "COPY" tool was used to transfer the files to the master floppy.

Now the really revealing bit. This junk data was found on the 720K version of LSL2, but the batch file is executing commands to copy the 360K version. I'm guessing that the final two lines prior to the cluster ending is indicating that it then went on to start copying files to drive B:. So it's just copied the 360K version to drive A, and then it starts copying to drive B:. My assumption therefore is that drive B: was the 3 1/2 " floppy drive and that this is when it started writing the 720K version of the game.

So how could a fragment of the 360K batch script for preparing the master floppies have ended up on the 720K disk? I think this is good evidence for the "dirty buffer" "over copy" bug that I'm thinking may have existed. DOS would have read the batch file in to memory quite recently, just prior to copying files to the floppies. So we know it was loaded in to memory, and we know that part of it ended up on one of the floppies.

A couple of new bits of information to add to this discussion. I ran a few questions past an ex-Sierra employee about what he remembered of the process for preparation of the master floppies and how these were then used to create the customer copies. He said that the floppies used were stated to be blank and were almost always pre-formatted. He said that this doesn't necessarily rule out the possibility that a previously used floppy was reused in some cases though. He said that after the set of disks was produced, it was then loaded in to a machine called a Formaster (they had a room full of these) and the data for the disks was held in that machine's internal memory. The machine then used the data in its memory to write out the copies of the disks. The same master set of floppies would therefore in effect be used for all copies, until such time that a bug was found. At that point the bug would be fixed and a new set of disks produced and loaded in to the Formasters.

He also said that they primarily used the standard DOS but that some developers used DR-DOS and 4DOS after those came out. I've checked the timelines of the releases of those flavours of DOS and I doubt that they would have been using them in 1988 when LSL2 was produced. 4DOS was released in February 1989, and the initial public release of DR-DOS was in May 1988. I suppose they could have jumped on the DR-DOS bandwagon immediately but it seems unlikely. And given that this thread started with a discussion of AGI specs and AGI assembler code, the dates of those AGI disks would probably pre-date LSL2 and therefore before DR-DOS. HWM said he thought it might have been LSL1 that the AGI fragments were on but that he wasn't sure. That would have been 1987 if it was the case.

The original game disks from 1988 have identifiers such as "MSDOS3.3" and "IBM 3.2" in the boot sector. I suppose that all this implies is that those versions of DOS were used to pre-format the disk.
« Last Edit: June 08, 2015, 05:34:09 PM by lance.ewing »

Offline HWM

Re: Fragments of the original AGI specs and interpreter code
« Reply #14 on: June 10, 2015, 06:02:22 PM »
It seems I've overlooked the most interesting file of the bunch I had saved before. I don't think I've looked past all the pages of the memory map (first part of the file) before, because otherwise I'd probably remembered, as it's pretty spectacular: C code of the interpreter complete with comments, programmer initials and everything. Based on the total size of it (210kb) it's probably from a different release than the code shared earlier. Also there's part of a batch file in there, creating a playable copy of the game(?):

Code: [Select]
copy agi.exe %2
copy *.ovl %2
setstr gameID %1 %2agi.exe
copy ..\src\agi.doc %2

Not sure about the "over copy" bug. It might be a combination of both (buffer + deleted data), since the attached 210kb of data seems a bit too much to be buffered. Just copying with (X)COPY would have taken the same amount of time as doing a full DISKCOPY then, and I suppose that's something that would've been noticed at one point.

I'm familiar with copies of the FAT being shattered all over the disk, but I don't know what causes it. I remember this one time, one of my disks somehow failed (corrupt filenames, file sizes of gigabytes). I inspected the disk with a disk editor and saw multiple copies of the FAT (or at least part of it) of one of my hard disk partitions there, which really struck me as odd. Again, no idea what causes it, but you'll see it on a lot of disks.

To conclude for now, I was able to check some of the SCI disks. There was some LISP-like code on the Codename: ICEMAN disk, but after some research it turned out to be macro code from a DOS application called BRIEF. So just in case, this is not SCI code:

Code: [Select]
.
;**

(macro right_side
(
(int num_cols
shift
)
(inq_window_size NULL num_cols shift)
(move_abs 0 (+ num_cols shift))
)
)

)
)


SMF 2.0.11 | SMF © 2015, Simple Machines
Simple Audio Video Embedder

Page created in 0.27 seconds with 21 queries.