C# AGI Interpreter

[lance.ewing]

Well I've finished the C# tutorial app that I've been working through on my phone over the past few weeks. It hasn't just been the tutorial though. As I've mentioned, I converted the VVVV game to C#, but I've also started to put together the beginnings of a C# AGI interpreter, something that I began about a week ago while I was still working through the tutorial.

What I've done is to fork the AGI/SCI Developer code base. I've forked it for a few reasons: One is because it might be dangerous for me to start contributing to the real thing at the moment. I figure that I can prototype something on the forked repository and then if I actually get something working, Collector can pull that back in to the main project. The other reason is so that I can make use of the AGI Library that comes with the AGI/SCI Developer. It already has code for loading the game data. It's structured for the purposes of the Editor, but it isn't far off what I'd need for the Interpreter. And it makes sense that if the Interpreter is to be integrated in to the editor that it uses the same classes for holding the game data so that the editor simply says "Here's the game data...  Interpret it!!"

Initially my plan was to port MEKA to C#, but that has evolved to some degree. What I'm now doing is looking over four sources of information as I build each component, those sources being the original AGI source code fragments, the original AGI documentation, the MEKA source code, and the AGDS documentation (i.e. the Russian project, which has quite a detailed explanation about the workings of the interpreter cycle).

I'm starting to the look at the AGI source code fragments very closely now and it looks like it has pretty much the complete animation code. There isn't anything missing in that area. I should be able to replicate the animation side of things quite well. It also includes the main top level interpreter cycle loop, but lacks the code for the execute logic routine. We can pretty  much guess what that does though, and there's quite a high percentage of the individual test and action commands represented in those original AGI source fragments as well. It's more complete than I originally thought. The C# project will be an object oriented representation of this logic.

I have yet to implement a single test or action command. I'm trying to lay the foundation first, which at the moment is working on the top level interpreter cycle and the animation code. Soon I'll be ready to dive in to the execution logic routine, but to be honest, its going to be weeks before I actually try to run it.

[lance.ewing]

Up to this point, I've only been using the existing AGI Library within AGI/SCI Developer for loading the resources, and the original AGI interpreter code fragments and MEKA source code for input in to the interpreter code.

I've taken the animation side of the interpreter code as far as I can without having any implemented test or action commands, so my attention is now focused on logic execution. For the animation code, I was able to use the View, Loop, and Cell classes from the AGI Library without any change at all. This won't be the case for the Logic class, as it is basically empty. All it has in it at present is the raw byte data loaded from the logic.

So this lead me to reassess my earlier thoughts about whether it was better to port JAGI to C# or MEKA. Although my memory is of MEKA working better than what I've seen of JAGI, the test and action commands are implemented in a very procedural way, and all the test and action functions make use of pointers. I'm now starting to think that perhaps porting the JAGI representation of a Logic resource to C# might be the way to go. I think it would require much less conversion. Once all the "Instructions" (as it calls them) are converted, I'd probably go through each one and compare to the other three sources of information to make sure the implementation is correct.

[Collector]

I would note that Gumby did a couple of classes for the backend for WORDS.TOK and OBJECT in his ResourceMgmt.cs. This functionality should probably be moved to the AGI library.

[gumby]

Yes, but the work that I did probably is not the most performant.  Though with current hardware it probably isn't a huge issue.  Hopefully it'll help on some level.

[Collector]

The loading of the WORDS.TOC with your Resource_AGI class was slow, but then loading an AGI game with Visual AGI's AGI Library is far slower than loading an SCI game with your SCI Resource class in Developer, too. You were going to look at it before you got buried by life.

[lance.ewing]

I think I've got to the point where I need to add Words and Object support in to the AGI Library. I've implemented about half of the action commands and half of the test commands in the execute Logic loop now. Obviously there are a few in there to do with inventory objects and words, so I guess I better decide how I go about adding these resource types in to the AGI Library. I think what I'm going to do is integrate the loading of the WORDS.TOK and OBJECT files in to the DecodeGame method in the Game class. The definition of the classes for holding the data for both I'll probably add to the Resource class in the same way that Picture, View, etc. are contained in there.

[Collector]

Don't forget that the library will also have to handle the creation of new and saving of modified word and object resources for the IDE.

[lance.ewing]

Yeah I'm definitely keeping that in mind, although I'll leave placeholders for "new" and "saving" for the time being. That's what I've done with the Logic class. There is code in there now to decode the raw data in to a usable object representation, where things like the messages and commands and their parameters are accessible as objects. This is what the interpreter is making use of for the execute Logic routine. But currently the Encode method in Logic is still completely empty. I have it on my mental list to add source code generation logic to the Logic class at some point, probably based on what I added to JAGI. I didn't like what JAGI had in there for its decompile method, so I wrote an alternative implementation that I will hopefully be able to use as the basis of the logic for the C# AGI Library decompile process. Compiling is obviously another thing on the list. Most of it comes after the interpreter is in some kind of working form though. I'm guessing I'm probably still a couple of months off having the interpreter running. It's steady going but slow going.

[Collector]

No rush. Obviously saving will not be needed until you are ready to merge your work into the main branch.

[lance.ewing]

Offtopic, but maybe not so much

Actually, that is quite off topic. I don't think you'll get much response if you leave the post under this topic. I'd like to reply to it properly, but I'm reluctant to do so under the "C# AGI Interpreter" topic as it will distract from the purpose of this topic. So if you don't mind, it would be best to move your last post to the original topic that you created here:

http://sciprogramming.com/community/index.php?topic=1499.msg8699#msg8699

That would logically continue the discussion in one place. I'm not certain if the author of a forum post is able to move it to another topic (I haven't tried it myself before), but if not, then perhaps a moderator will be able to do it for us. Or if all else fails, I guess the manual approach of copy and pasting in to a new post under the original topic.

[lance.ewing]

I'm making good progress with the C# AGI interpreter. I've implemented all but two of the test commands & nearly three quarters of the action commands. I decided it might be time to see what would happen if I clicked the run button. KQ3 was my test game as it is the only one that I tried that the AGI Library was able to decode all the resources of. There seem to be some pre-existing issues in the AGI Library that I'll need to look at once I've got more of the interpreter implemented. For now I'm going to use KQ3.

As I was expecting, there were a few bugs in my code, but I worked through fixing those and it's now running through the first three screens of the title & credit sequence pretty much as in the original. It gets as far as the first scene in the intro part but no further because I haven't implemented the print commands. I also haven't implemented user input  of any kind, so no input line or menu yet. The chickens are doing their wandering and pecking thing in their pen.

I'm in the middle of implementing the text windows and print commands at the moment.

[Crispin]

Offtopic, but maybe not so much

Actually, that is quite off topic.

Yes, you're right - I moved it to proper topic. I was doing few things at once so I messed up.
Anyway, I'm glad to see that C# AGI interpreter is going well. Althought... C#, .Net & Mono are not my flavours.

[Collector]

What issues were you running into with the library?

[lance.ewing]

To be honest, I hadn't looked any further at the issues other than to see that it was complaining about certain resources, so I quickly moved on to the next game until I found one that it was 100% happy with.

But now that I look at the output from KQ1, it seems it could be data issues with the DIR files. AGI Studio doesn't like the same sound resources, so I guess that is a good indication that there is some unused garbage data in there, like junk DNA. The game almost certainly doesn't use those resources then. I've probably forgotten that we get a few dodgy looking DIR entries every now and then.

Code: [Select]

...
SKIPPING: Resource header position 0x20E2F is out of bounds. Skipping resource SndDir\34!
SKIPPING: Resource header position 0x20E8F is out of bounds. Skipping resource SndDir\35!
SKIPPING: Resource header position 0x20EED is out of bounds. Skipping resource SndDir\36!
SKIPPING: Resource header position 0x2126B is out of bounds. Skipping resource SndDir\37!
Successfully read 22 resources from 'SndDir'.
SKIPPING: Could not decode resource. The error given was:
Index was outside the bounds of the array.
Successfully read 90 resources from 'LogDir'.
The operation completed with 5 errors.

So perhaps there are no real issues, which is great news. I think I'm going to give the interpreter a go on a few more games then.  :-)

[lance.ewing]

The issues with KQ2 look a bit more serious:

Code: [Select]

SKIPPING: Could not decode resource. The error given was:
Decode error in command 0xF6 at position 507.
Command must have at least two data bytes, found 0.
*** Opened vol.1: 0x2E82D bytes.
*** Opened vol.0: 0xEB92 bytes.
SKIPPING: Could not decode resource. The error given was:
Decode error in command 0xF6 at position 1765.
Command has a wrong number of data bytes.
SKIPPING: Could not decode resource. The error given was:
Decode error in command 0xF3 at position 0.
Command has too many data bytes. Expected 0, found 2.
Successfully read 110 resources from 'PicDir'.
SKIPPING: Could not decode resource. The error given was:
Source array was not long enough. Check srcIndex and length, and the array's lower bounds.

Seems to be complaining about picture commands.