C# AGILE

[AGKorson]

Also, is your source code available for your C# interpreter? I'd be happy to take a look at any sections where you have questions/concerns; I might be able to find issues quickly based on my decompilation efforts.

[lance.ewing]

Also, is your source code available for your C# interpreter? I'd be happy to take a look at any sections where you have questions/concerns; I might be able to find issues quickly based on my decompilation efforts.

The project is currently a fork of Collector's AGI/SCI Developer repo in Bitbucket. That repo is described as a merger of SCI Developer and Visual AGI. It has a separate AGI Library that Visual AGI uses to load and save resources, although it didn't have anything to deal with SOUND or LOGIC resources when I started out with the Interpreter, and it didn't have anything for loading AGI V3 games either. The reason I forked Collector's repo is because one of the end goals is that it would be possible to run an AGI game from within the Visual AGI part of the IDE, so it is built on top of that AGI Library.

So having said all that, Collector's repo is currently private, and my fork of his repo is also private. If you create a Bitbucket account though, or already have one, then I can share my repo with you so that you can take a look. It would be very helpful actually. It would be good to get your feedback on what you see in there. I know there are still a number of outstanding issues. I was playing through SQ2 last night (nearly half way through I think) and spotted another four issues I need to take a look at. There are also several issues in KQ4.

[lance.ewing]

If you have verified that it works correctly with the original DOS interpreter, I would love to get a copy of that interpreter file; I've decompiled all AGI versions that I have, including all v3 versions. I can confirm that only 3.002.102 and above use flag 20. I've examined the code in the 3.002.086 version I have, and there is nothing there that I can see that would change loops (without specific code in the logic), so I would love to know if the interpreter file I have is authentic (the date/time stamp is 8/31/1988 12:38PM). Would you mind sharing your file?

The date/time stamp of my one is 7/25/1988 12:43PM, as it currently appears on my harddisk. The AGI file is 40866 bytes. I can't recall now where I found this one, but I've had it for a while I think. I can confirm that the game definitely handles the loop changing even though the VIEW has 5 loops.

Also, I don't have access to my original disks (I'm on temporary assignment for work away from home for several months). Could I ask for a copy of the game files for your KQ4 version as well?  Thx!

PM sent.

[AGKorson]

I've checked this now, and flag 20 is not set. So I guess that agrees with the fact that only 3.002.102 and above use flag 20 for controlling this.

Version 3.002.086 must be doing something similar but not controlled by flag 20.

I found it! It's a bug that's unique to version 3.002.086. Here's the code in v2.917 (which is identical in all versions up to 3.002.086:

Code: [Select]

   mov     [bp+newloop], 4                    ; set newloop value = 4(means no loop change)
   test    word ptr [si+flags], FIXED_LOOP    ; fixed loop?
   jnz     short setTheLoop                   ; if fixed, don't change loop
   cmp     byte ptr [si+loop_count], 2        ; loop count
   jz      short autoloop_sm                  ; if 2 loops, jump to small-loop selector
   cmp     byte ptr [si+loop_count], 3
   jnz     short autoloop_lg                  ; if three, also use small loop selector
                                              ; but if NOT three, go on to further check
autoloop_sm:
   mov     al, [si+obj_dir]                   ; use object direction
   sub     ah, ah
   mov     bx, ax                             ; to
   mov     al, loop_small[bx]                 ; convert direction to correct loop
   jmp     short changeLoopVal

autoloop_lg:
   cmp     byte ptr [si+loop_count], 4        ; if loop count != 4
   jnz     short setTheLoop                   ; don't change the loop
   mov     al, [si+obj_dir]                   ; if loop count == 4
   sub     ah, ah                             ; use object direction
   mov     bx, ax                             ; to
   mov     al, loop_large[bx]                 ; convert direction to correct loop
changeLoopVal:
   mov     [bp+newloop], al
setTheLoop:
...   ; change loop if newloop value has changed (!= 4)
In 3.002.0086, they made one very small change (marked with ***):

Code: [Select]

   mov     [bp+newloop], 4                    ; set newloop value = 4(means no loop change)
   test    word ptr [si+flags], FIXED_LOOP    ; fixed loop?
   jnz     short setTheLoop                   ; if fixed, don't change loop
   cmp     byte ptr [si+loop_count], 2        ; loop count
   jz      short autoloop_sm                  ; if 2 loops, jump to small-loop selector
   cmp     byte ptr [si+loop_count], 3
   jnz     short autoloop_lg                  ; if three, also use small loop selector
                                              ; but if NOT three, go on to further check
autoloop_sm:
   mov     al, [si+obj_dir]                   ; use object direction
   sub     ah, ah
   mov     bx, ax                             ; to
   mov     al, loop_small[bx]                 ; convert direction to correct loop
   jmp     short changeLoopVal

autoloop_lg:
   cmp     byte ptr [si+loop_count], 4        ; if loop count ***<*** 4
   ***jb***      short setTheLoop             ; don't change the loop
   mov     al, [si+obj_dir]                   ; if loop count ***>*** 4
   sub     ah, ah                             ; use object direction
   mov     bx, ax                             ; to
   mov     al, loop_large[bx]                 ; convert direction to correct loop
changeLoopVal:
   mov     [bp+newloop], al
setTheLoop:
...   ; change loop if newloop value has changed (!=4)
The change in that test from 'jnz' to 'jb' has the effect of always changing the loop no matter how many loops are in the view. In version 3.002.098 (not 3.002.102, as I previously thought), the check for flag f20 was added which fixed the problem.

The upshot of this is that for all versions up to 3.002.086, objects assigned a view with more than 4 loops will not update loop based on direction; in version 3.002.086 objects will update loop based on direction no matter how many loops there are; and in versions beyond 3.002.086 objects with more than 4 loops will only update loop if flag f20 is TRUE.

[lance.ewing]

Thanks for solving that mystery. That definitely explains it then. Looks like I will have to add specific version checks in there to make it truly compatible.

[AGKorson]

Thanks for solving that mystery. That definitely explains it then. Looks like I will have to add specific version checks in there to make it truly compatible.

The one thing I've learned after all this time studying AGI is that Sierra was terrible at error checking and that a lot of their code (in the interpreter) was a bit sloppy in handling non-standard data. When opening Sierra games you gotta check for a whole bunch of things that can cause problems.

For example, when I loaded KQ4 yesterday, I found there are four resources (picture 150, 151 and view 198, 199) that point to non-existent VOL files. And there are five pictures (2, 10, 15, 102, 115) that have bad data for pixels (using a Y value of 168 occurs a bunch of times in all those pictures). Sierra's interpreter will happily overflow the picture buffer without complaining (the stuff that gets overwritten is whatever is beyond the picture buffer in memory, which is not even allocated to the program!). So they display with no visible effects, but if you want to draw them in a new interpreter or editor, you have to account for that.

[lance.ewing]

For example, when I loaded KQ4 yesterday, I found there are four resources (picture 150, 151 and view 198, 199) that point to non-existent VOL files. And there are five pictures (2, 10, 15, 102, 115) that have bad data for pixels (using a Y value of 168 occurs a bunch of times in all those pictures). Sierra's interpreter will happily overflow the picture buffer without complaining (the stuff that gets overwritten is whatever is beyond the picture buffer in memory, which is not even allocated to the program!). So they display with no visible effects, but if you want to draw them in a new interpreter or editor, you have to account for that.

Yes, I've come across both of those issues in KQ4 already. For resources that point to non-existent VOL files, I think we can just ignore that. Clearly they're unused resources, and logic scripts must not be referring to them. Some of the other games have that issue as well.

For those pictures where the 0xF7 relative draw tool is drawing down to Y = 168, I'm not 100% sure what to do. If I remove the validation check, then they load fine and render fine in my Interpreter, but then that means we're allowing an original Sierra game to violate rules that our own editors enforce. Sierra's original PICTURE editor must have allowed them to draw down to 168, perhaps as a result of a bug maybe? Given this issue is in KQ4, maybe it was one of their later PICTURE editors. This issue doesn't seem to be in any other game that I've noticed so far.

I guess what I need to do is put some sort of option for disabling that validation into the Picture part of the AGI Library. Maybe original games can use the AGI Library with validation turned off, whereas the picture tool in Visual AGI would have to run with it turned on, although it probably doesn't need to, because if it has been written properly, it won't let such data be created.

[OmerMor]

Sierra's original PICTURE editor must have allowed them to draw down to 168, perhaps as a result of a bug maybe? Given this issue is in KQ4, maybe it was one of their later PICTURE editors. This issue doesn't seem to be in any other game that I've noticed so far.

I have 2 versions of Sierra's PICTURE editor: PIC.EXE (1987-02-23) and PE.EXE (1987-05-28).
I did not check them out yet, but you are welcome to do so: I just released my stash of AGI tools in this thread: http://sciprogramming.com/community/index.php?topic=1814.0

[AGKorson]

For those pictures where the 0xF7 relative draw tool is drawing down to Y = 168, I'm not 100% sure what to do. If I remove the validation check, then they load fine and render fine in my Interpreter, but then that means we're allowing an original Sierra game to violate rules that our own editors enforce. Sierra's original PICTURE editor must have allowed them to draw down to 168, perhaps as a result of a bug maybe? Given this issue is in KQ4, maybe it was one of their later PICTURE editors. This issue doesn't seem to be in any other game that I've noticed so far.

As I mentioned earlier, in the interpreter, the position is converted to a linear offset in the picture memory buffer (pos = 160*y+x). The interpreter doesn't bother checking either X or Y; so when the offset exceeds the actual buffer size, AGI just sets that byte in memory even thought it's a buffer overflow. But since the picture buffer is the very last hunk of memory that is allocated, it doesn't overwrite any AGI data. So it looks just fine in AGI.

In WinAGI, I just do a check every time a pixel is plotted. If the linear offset exceeds the buffer size, I just ignore it. That way the picture will be rendered the same as in AGI.

[AGKorson]

Sierra's original PICTURE editor must have allowed them to draw down to 168, perhaps as a result of a bug maybe? Given this issue is in KQ4, maybe it was one of their later PICTURE editors. This issue doesn't seem to be in any other game that I've noticed so far.

I have 2 versions of Sierra's PICTURE editor: PIC.EXE (1987-02-23) and PE.EXE (1987-05-28).
I did not check them out yet, but you are welcome to do so: I just released my stash of AGI tools in this thread: http://sciprogramming.com/community/index.php?topic=1814.0

Thx Omer. Maybe I will try to decompile these to see it there is a bug that allowed a Y value of 168 to slip in.

[lance.ewing]

I was playing through SQ2 last night (nearly half way through I think) and spotted another four issues I need to take a look at.

Actually there were five issues I had spotted in SQ2 at the nearly halfway through mark. All now fixed, so I'll continue playing through to the end now.

[lance.ewing]

Actually there were five issues I had spotted in SQ2 at the nearly halfway through mark. All now fixed, so I'll continue playing through to the end now.

I've finished playing SQ2 through to the end. I did notice a couple of minor issues that I need to take a look at. The text in one of the text windows was wrapped differently. I thought I had that working properly, but I'll need to take another look and see why this particular one isn't right. And there was one sentence that it didn't understand that the original appears to understand, which has made me realise I have't implemented ignored words yet. Not sure how I've got so far without that.

AGKorson is helping me with an issue in KQ4 to do with effective Y calculation of fixed priority objects that I will hopefully have resolved later today. Thanks Andrew for identifying the issue.

Assuming I fix the above issues over the next day, then the main outstanding issue I have at the moment is in MH2. For some reason it hangs on the WAIT icon after leaving the passage way that leads to Tad Timov's office to return outside. The scripts seem to be stuck in some kind of loop, which means whatever it is waiting for the Interpreter to do, it obviously isn't doing.

Next game I want to try playing through to the end is KQ4.

[Collector]

Looks like it is the AGI3 games where you are finding issues.

[lance.ewing]

It's a mixture actually. All the early issues I found in KQ1, because that was the first game I was trying to play through. Then above I've recently mentioned seven issues I found while playing SQ2, which is AGI v2. But I have found a lot of issues in the AGI v3 games as well. There are a lot of AGI v2 games I haven't played through much of, so there could be other issues in there. I did spend quite a bit of time looking at DDP and MUMG because they do some unusual things, unusual when compared with the core set of AGI v2 games. There were quite a few issues I worked through with those two games.

[lance.ewing]

And there was one sentence that it didn't understand that the original appears to understand, which has made me realise I have't implemented ignored words yet. Not sure how I've got so far without that.

So turns out I have implemented ignored words already. The bit that isn't working is recognising the longest "word" when it is trying to match the input line with recognised word numbers. This recognition is breaking down when an ignored word is involved. So in the sentence "cut glass with cutter", it is ignoring "with" as expected, but is then matching "glass cutter" as a whole word, because that has a word number, which is probably a synonym of "cutter", and having ignored "with", it now thinks "glass" and "cutter" are next to each other. I can think of a quick way of fixing this already. Will try that now.