Community

AGI Programming => AGI Development Tools => Topic started by: lance.ewing on December 04, 2016, 03:56:08 PM

Title: C# AGI Interpreter
Post by: lance.ewing on December 04, 2016, 03:56:08 PM
Well I've finished the C# tutorial app that I've been working through on my phone over the past few weeks. It hasn't just been the tutorial though. As I've mentioned, I converted the VVVV game to C#, but I've also started to put together the beginnings of a C# AGI interpreter, something that I began about a week ago while I was still working through the tutorial.

What I've done is to fork the AGI/SCI Developer code base. I've forked it for a few reasons: One is because it might be dangerous for me to start contributing to the real thing at the moment. I figure that I can prototype something on the forked repository and then if I actually get something working, Collector can pull that back in to the main project. The other reason is so that I can make use of the AGI Library that comes with the AGI/SCI Developer. It already has code for loading the game data. It's structured for the purposes of the Editor, but it isn't far off what I'd need for the Interpreter. And it makes sense that if the Interpreter is to be integrated in to the editor that it uses the same classes for holding the game data so that the editor simply says "Here's the game data...  Interpret it!!"

Initially my plan was to port MEKA to C#, but that has evolved to some degree. What I'm now doing is looking over four sources of information as I build each component, those sources being the original AGI source code fragments, the original AGI documentation, the MEKA source code, and the AGDS documentation (i.e. the Russian project, which has quite a detailed explanation about the workings of the interpreter cycle).

I'm starting to the look at the AGI source code fragments very closely now and it looks like it has pretty much the complete animation code. There isn't anything missing in that area. I should be able to replicate the animation side of things quite well. It also includes the main top level interpreter cycle loop, but lacks the code for the execute logic routine. We can pretty  much guess what that does though, and there's quite a high percentage of the individual test and action commands represented in those original AGI source fragments as well. It's more complete than I originally thought. The C# project will be an object oriented representation of this logic.

I have yet to implement a single test or action command. I'm trying to lay the foundation first, which at the moment is working on the top level interpreter cycle and the animation code. Soon I'll be ready to dive in to the execution logic routine, but to be honest, its going to be weeks before I actually try to run it.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on December 12, 2016, 03:09:30 PM
Up to this point, I've only been using the existing AGI Library within AGI/SCI Developer for loading the resources, and the original AGI interpreter code fragments and MEKA source code for input in to the interpreter code.

I've taken the animation side of the interpreter code as far as I can without having any implemented test or action commands, so my attention is now focused on logic execution. For the animation code, I was able to use the View, Loop, and Cell classes from the AGI Library without any change at all. This won't be the case for the Logic class, as it is basically empty. All it has in it at present is the raw byte data loaded from the logic.

So this lead me to reassess my earlier thoughts about whether it was better to port JAGI to C# or MEKA. Although my memory is of MEKA working better than what I've seen of JAGI, the test and action commands are implemented in a very procedural way, and all the test and action functions make use of pointers. I'm now starting to think that perhaps porting the JAGI representation of a Logic resource to C# might be the way to go. I think it would require much less conversion. Once all the "Instructions" (as it calls them) are converted, I'd probably go through each one and compare to the other three sources of information to make sure the implementation is correct.
Title: Re: C# AGI Interpreter
Post by: Collector on December 12, 2016, 10:42:12 PM
I would note that Gumby did a couple of classes for the backend for WORDS.TOK and OBJECT in his ResourceMgmt.cs. This functionality should probably be moved to the AGI library.
Title: Re: C# AGI Interpreter
Post by: gumby on December 14, 2016, 07:20:32 PM
Yes, but the work that I did probably is not the most performant.  Though with current hardware it probably isn't a huge issue.  Hopefully it'll help on some level.
Title: Re: C# AGI Interpreter
Post by: Collector on December 14, 2016, 10:52:33 PM
The loading of the WORDS.TOC with your Resource_AGI class was slow, but then loading an AGI game with Visual AGI's AGI Library is far slower than loading an SCI game with your SCI Resource class in Developer, too. You were going to look at it before you got buried by life.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on December 24, 2016, 04:52:31 AM
I think I've got to the point where I need to add Words and Object support in to the AGI Library. I've implemented about half of the action commands and half of the test commands in the execute Logic loop now. Obviously there are a few in there to do with inventory objects and words, so I guess I better decide how I go about adding these resource types in to the AGI Library. I think what I'm going to do is integrate the loading of the WORDS.TOK and OBJECT files in to the DecodeGame method in the Game class. The definition of the classes for holding the data for both I'll probably add to the Resource class in the same way that Picture, View, etc. are contained in there.
Title: Re: C# AGI Interpreter
Post by: Collector on December 24, 2016, 09:38:03 AM
Don't forget that the library will also have to handle the creation of new and saving of modified word and object resources for the IDE.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on December 24, 2016, 12:13:00 PM
Yeah I'm definitely keeping that in mind, although I'll leave placeholders for "new" and "saving" for the time being. That's what I've done with the Logic class. There is code in there now to decode the raw data in to a usable object representation, where things like the messages and commands and their parameters are accessible as objects. This is what the interpreter is making use of for the execute Logic routine. But currently the Encode method in Logic is still completely empty. I have it on my mental list to add source code generation logic to the Logic class at some point, probably based on what I added to JAGI. I didn't like what JAGI had in there for its decompile method, so I wrote an alternative implementation that I will hopefully be able to use as the basis of the logic for the C# AGI Library decompile process. Compiling is obviously another thing on the list. Most of it comes after the interpreter is in some kind of working form though. I'm guessing I'm probably still a couple of months off having the interpreter running. It's steady going but slow going.
Title: Re: C# AGI Interpreter
Post by: Collector on December 24, 2016, 06:38:37 PM
No rush. Obviously saving will not be needed until you are ready to merge your work into the main branch.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 11, 2017, 04:10:04 PM
Offtopic, but maybe not so much

Actually, that is quite off topic. I don't think you'll get much response if you leave the post under this topic. I'd like to reply to it properly, but I'm reluctant to do so under the "C# AGI Interpreter" topic as it will distract from the purpose of this topic. So if you don't mind, it would be best to move your last post to the original topic that you created here:

http://sciprogramming.com/community/index.php?topic=1499.msg8699#msg8699

That would logically continue the discussion in one place. I'm not certain if the author of a forum post is able to move it to another topic (I haven't tried it myself before), but if not, then perhaps a moderator will be able to do it for us. Or if all else fails, I guess the manual approach of copy and pasting in to a new post under the original topic.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 11, 2017, 04:15:37 PM
I'm making good progress with the C# AGI interpreter. I've implemented all but two of the test commands & nearly three quarters of the action commands. I decided it might be time to see what would happen if I clicked the run button. KQ3 was my test game as it is the only one that I tried that the AGI Library was able to decode all the resources of. There seem to be some pre-existing issues in the AGI Library that I'll need to look at once I've got more of the interpreter implemented. For now I'm going to use KQ3.

As I was expecting, there were a few bugs in my code, but I worked through fixing those and it's now running through the first three screens of the title & credit sequence pretty much as in the original. It gets as far as the first scene in the intro part but no further because I haven't implemented the print commands. I also haven't implemented user input  of any kind, so no input line or menu yet. The chickens are doing their wandering and pecking thing in their pen.

I'm in the middle of implementing the text windows and print commands at the moment.
Title: Re: C# AGI Interpreter
Post by: Crispin on January 11, 2017, 07:32:02 PM
Offtopic, but maybe not so much

Actually, that is quite off topic.

Yes, you're right - I moved it to proper topic. I was doing few things at once so I messed up.
Anyway, I'm glad to see that C# AGI interpreter is going well. Althought... C#, .Net & Mono are not my flavours.
Title: Re: C# AGI Interpreter
Post by: Collector on January 11, 2017, 10:27:19 PM
What issues were you running into with the library?
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 12, 2017, 01:51:41 PM
To be honest, I hadn't looked any further at the issues other than to see that it was complaining about certain resources, so I quickly moved on to the next game until I found one that it was 100% happy with.

But now that I look at the output from KQ1, it seems it could be data issues with the DIR files. AGI Studio doesn't like the same sound resources, so I guess that is a good indication that there is some unused garbage data in there, like junk DNA. The game almost certainly doesn't use those resources then. I've probably forgotten that we get a few dodgy looking DIR entries every now and then.

Code: [Select]
...
SKIPPING: Resource header position 0x20E2F is out of bounds. Skipping resource SndDir\34!
SKIPPING: Resource header position 0x20E8F is out of bounds. Skipping resource SndDir\35!
SKIPPING: Resource header position 0x20EED is out of bounds. Skipping resource SndDir\36!
SKIPPING: Resource header position 0x2126B is out of bounds. Skipping resource SndDir\37!
Successfully read 22 resources from 'SndDir'.
SKIPPING: Could not decode resource. The error given was:
Index was outside the bounds of the array.
Successfully read 90 resources from 'LogDir'.
The operation completed with 5 errors.

So perhaps there are no real issues, which is great news. I think I'm going to give the interpreter a go on a few more games then.  :-)
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 12, 2017, 02:02:54 PM
The issues with KQ2 look a bit more serious:

Code: [Select]
SKIPPING: Could not decode resource. The error given was:
Decode error in command 0xF6 at position 507.
Command must have at least two data bytes, found 0.
*** Opened vol.1: 0x2E82D bytes.
*** Opened vol.0: 0xEB92 bytes.
SKIPPING: Could not decode resource. The error given was:
Decode error in command 0xF6 at position 1765.
Command has a wrong number of data bytes.
SKIPPING: Could not decode resource. The error given was:
Decode error in command 0xF3 at position 0.
Command has too many data bytes. Expected 0, found 2.
Successfully read 110 resources from 'PicDir'.
SKIPPING: Could not decode resource. The error given was:
Source array was not long enough. Check srcIndex and length, and the array's lower bounds.

Seems to be complaining about picture commands.
Title: Re: C# AGI Interpreter
Post by: Collector on January 12, 2017, 08:57:49 PM
Trying to open King's Quest 2 version 2.1, int 2.411 with the version of the library in Developer I get one error:

Code: [Select]
*** Opened vol.2: 0x3C890 bytes.
SKIPPING: Could not decode resource. The error given was:
Decode error in command 0xF6 at position 507.
Command must have at least two data bytes, found 0.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 13, 2017, 06:58:23 PM
I've realised I wasn't running against a clean install of KQ2.  :-[ It had a couple of extra pictures added to it that had been extracted from the AGI v1 KQ2 (which was part of some tests I did to prove that they were the same format as AGI v2 pictures - Interesting that AGI Studio renders them fine though). So we can discount those two pictures for now at least.

So I think I'm back to a clean install and I'm now getting the one error that you are also getting. I've tweaked the SKIPPING messages for decode failures so that it includes the resource number. From this I can see that it doesn't like picture 12. AGI Studio is quite happy with it though, but I extracted it and then loaded it in to PICEDIT (1.3M6) and it didn't like it, in fact it ended up hanging. I had to use the Windows Task Manager to kill it. Tried it in Linux as well and the same thing happened. Had to kill the process. But PICEDIT 1.2.1 (the first of the Java versions) loaded the picture without any issues.

I think I need to take a look at position 507 and see what is going on.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 13, 2017, 07:04:43 PM
It would appear that there is indeed an 0xF6 code without any subsequent data bytes. That's kind of redundant then. It might imply that the person that created the picture switched to the absolute line tool but didn't actually plot any points. AGI Studio and PICEDIT 1.2.1 seem to handle this fine and don't require at least one point, but the AGI Library and the newer PICEDIT versions appear to expect at least one point.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 13, 2017, 07:10:49 PM
Fixed that by commenting out the validation check in the AGI Library. Now it's got to position 980 in the same picture and complaining that a 0xF7 (relative line) doesn't have at least one point.

Edit: I think it's going to need more that just removing those checks during the decode. The drawing routine also needs to change to cope with no points for those actions.
Title: Re: C# AGI Interpreter
Post by: Collector on January 13, 2017, 09:03:58 PM
Since this is with an official Sierra game I would say that the library needs to handle this in a way that will allow the resource to load.

You mention AGI Studio. It has no picture editor. Are you talking about its viewer?

Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 14, 2017, 04:43:23 AM
Yeah, I agree. I'm going to try to make it work with that picture this evening. I confirmed by walking ego in to room 12 that the interpreter doesn't like that picture after having ignored/commented out the errors about there being no point data. It then fell over trying to draw the picture for the room when entering that room.

Yeah, the Picture viewer in AGI Studio is what I was referring to. It draws the picture without any problems, which means that it doesn't require there to be any points at all for the line drawing commands. This is probably because it would have been tested with all the games in its early days and things like this discovered. It is probably why version 1.2.1 of PICEDIT (and presumably the old DOS version as well) would handle it. I probably forgot about this when making the more enhanced version of PICEDIT.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 14, 2017, 05:12:07 PM
I was in two minds about how to fix this. I started going down the path of retaining the point-less (pun intended) line commands in the data and add support for decoding, drawing, and encoding such cases. But then I started thinking that maybe the decode should just ignore the whole picture command. So that's what I'm doing now. If an absolute or relative line command doesn't have any points, then it gets filtered out (I could do the same for other tool commands as well). This would mean that if the AGI Library were used to simply load, decode, encode and then save, the end result would be different from what it started with. Visually the pictures would be identical, but the end result would have less data due to the do nothing commands having been removed. That feels okay to me. The purest in me wouldn't want someone to save picture 12 back in to KQ2 thereby removing such junk data, but I doubt anyone is going to do that unless they are making some kind of KQ2 mod game. For fan games, I think it is fine to filter out junk data.

Regardless, this approach has fixed the rendering of KQ2 picture 12 in the C# AGI interpreter.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 14, 2017, 05:33:32 PM
I've also noticed that what PICEDIT calls the Brush command hasn't been implemented yet in the AGI Library. This might be why the interpreter isn't able to load SQ2 at the moment. I recall that that game used a lot of that particular picture command.

It also doesn't support AGI V3 games. I tried loading one and it asked me if I was sure it was an AGI V2 game. So that is currently ruling out KQ4, GR, and the MH games. It is not too involved to add AGI V3 support to the AGI Library, but that's one of the things I'll leave until after I've got the interpreter working nicely with the AGI V2 games.
Title: Re: C# AGI Interpreter
Post by: Collector on January 15, 2017, 10:48:56 AM
I had never tried to load a AGI3 game with the library before, so I had missed this.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 16, 2017, 02:18:44 PM
I'm making further good progress on the interpreter. The message boxes are now fully implemented, so the print, print.v, print.at, and print.at.v commands are all working as they should. The display commands were already working prior to them. As far as text features go, there is still the menu system, the status bar, and the inventory screen that need to be built. None of that is there yet.

I have also started to add some user input. It now reacts to the user skipping an intro to get in to the game (i.e. have.key is working) and the arrow keys are also working for moving ego around. So its now possible to get in to KQ1 and KQ2 to walk around their outside areas from room to room, and for KQ3 to walk around the rooms of the castle where you start.

There is no said command input yet though, so it isn't possible to do things like enter a building by opening the door. I think I'm going to work on that next, as it opens up a whole lot more of what you can do within a game. The menu, status bar, and inventory screens are nice to haves in comparison.   :)
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 21, 2017, 03:04:17 PM
I now have the user input line working, so I can "look castle", "show carrot", etc. Actually I picked up a carrot in the garden in KQ1, opened the goat pen, showed the goat the carrot, he started following me, and then I took him to the bridge with the troll and he took care of him for me. So parsing seems to be working well. I've also got the status line working now.

I then moved on to the set.key and controller commands but I realised in the process that I needed to refactor the way I'm handling key events. The controller checks are working to a degree, but it's not ideal. So I'm partway through fixing that up at the moment.

I've noticed a few bugs as I was walking around KQ1 that I'm planning to take a look at after that, and then I might move on to either the inventory screen or the menu.
Title: Re: C# AGI Interpreter
Post by: Collector on January 21, 2017, 08:34:45 PM
Sounds like you are making good progress. Perhaps too soon, but one thought just occurred to me. For a new interpreter it would be a good idea to have the save directory default to the user space, say "C:\Users\<User_Name\Saved Games\<Game_Name>".
Title: Re: C# AGI Interpreter
Post by: Kawa on January 21, 2017, 08:55:32 PM
Oh yes please, more games should use that location instead of mucking up My Documents. I have some minimal C# code if you need it, since last I checked System.Environment's locations list didn't have it.

Because you should never hardcode such things. Profiles can be moved, specific folders can be moved, they can be localized... Is it "C:\Users\Kawa\Saved Games", "C:\Gebruikers\Kawa\Opgeslagen Spellen", "D:\Saves", or are you even running on a Windows with that folder at all? Best to try to retrieve the Saved Games location, then use System.Environment.GetFolderPath to grab AppData\Roaming as a fallback if it throws up. That way, Mono will automatically return ~/.config too.
Title: Re: C# AGI Interpreter
Post by: Collector on January 21, 2017, 11:23:44 PM
I would assume that Lance would simply grab it from the environment variable. Not even sure why you are even bringing it up.
Title: Re: C# AGI Interpreter
Post by: Kawa on January 22, 2017, 06:48:43 AM
There's an environment variable for that thing?

Edit: not on my system there isn't! Unless you meant the user profile path, which only answers half the question.
Title: Re: C# AGI Interpreter
Post by: MusicallyInspired on January 22, 2017, 09:43:06 AM
Am I the only one who hates it when games save games in user profile folders? I'd rather they just stay in the interpreter folder.
Title: Re: C# AGI Interpreter
Post by: Kawa on January 22, 2017, 10:01:35 AM
To be fair, if the application is running from Program Files, it may not be allowed to save in its own startup location.
Title: Re: C# AGI Interpreter
Post by: MusicallyInspired on January 22, 2017, 10:39:55 AM
Who actually uses Program Files for things like this?
Title: Re: C# AGI Interpreter
Post by: troflip on January 22, 2017, 11:05:47 AM
Oh yes please, more games should use that location instead of mucking up My Documents. I have some minimal C# code if you need it, since last I checked System.Environment's locations list didn't have it.

You can get the "SaveGame" folder by PInvoke'ing the native apis (SHGetKnownFolderPath with FOLDERID_SavedGames (which only works on Vista and later) ):
http://stackoverflow.com/questions/3795023/downloads-folder-not-special-enough/3795159#3795159

Title: Re: C# AGI Interpreter
Post by: Kawa on January 22, 2017, 12:40:33 PM
That's what I do, actually.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 22, 2017, 12:54:58 PM
This whole area is yet another thing that I know nothing about. All my commercial experience is with writing server side Java code for web applications that run on Linux. So whenever I dabble with desktop apps, I'm stackoverflowing things most of the way through.

So I'll let you guys reach a consensus on this and I'll go with whatever you think is best practice. If you do have some sample C# code for when I need it, that would be great.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 22, 2017, 01:05:57 PM
Sounds like you are making good progress. Perhaps too soon, but one thought just occurred to me. For a new interpreter it would be a good idea to have the save directory default to the user space, say "C:\Users\<User_Name\Saved Games\<Game_Name>".

Yeah, it probably is too soon at the moment. Saving games is one of those things towards the end of the list, along with Sound. I might even tackle sound before saved games. Sound in C# is obviously another thing I know nothing about, so that will be interesting  :)

The saved game topic does raise an interesting question though. I guess I should probably be saving in the original AGI saved game format. Hadn't really thought about that until now.

I got the controllers and set.key working nicely last night. This evening I might look at a couple of bugs and then maybe at implementing the inventory screen.
Title: Re: C# AGI Interpreter
Post by: Collector on January 22, 2017, 01:19:40 PM
Who actually uses Program Files for things like this?

This was an issue with Sunlight Games when I did the Gold Rush! package for them. They wanted the default for to be installed in the %ProgramFiles% directory. They only relented after I explained the permissions problems with legacy games in a system folder. You have to think of the range of types of users that may install your game. If it is installed inside of the %ProgramFiles% directory with UAC on the save games would just end up in the Virtual Store, not the game's folder. I would also note that the Save Game folder in the user space is best for a multi-user situation so that different users can have their own save games that won't be over written by another user.

The Save Games folder is where ScummVM, AGS and most newer game engines default. It is simply modern best practices. Though if this is done with this new interpreter I would not mind a configuration setting or switch to allow another location. ScummVM does allow this.

But all of this is far too early for much consideration at this point in the development of the new interpreter.
Title: Re: C# AGI Interpreter
Post by: Collector on January 22, 2017, 02:05:31 PM
This whole area is yet another thing that I know nothing about. All my commercial experience is with writing server side Java code for web applications that run on Linux. So whenever I dabble with desktop apps, I'm stackoverflowing things most of the way through.

So I'll let you guys reach a consensus on this and I'll go with whatever you think is best practice. If you do have some sample C# code for when I need it, that would be great.

Many of the special folders paths can be obtained with just:

System.Environment.GetEnvironmentVariable("USERPROFILE");

It is good enough for getting the main special folders such as the user profile, documents, temp, etc., but it does not get all special folders. Here is a CodeProject page with sample code in C# for getting *all* special folders in .NET. https://www.codeproject.com/articles/878605/getting-all-special-folders-in-net
Title: Re: C# AGI Interpreter
Post by: Collector on January 22, 2017, 02:10:34 PM
Yeah, it probably is too soon at the moment. Saving games is one of those things towards the end of the list, along with Sound. I might even tackle sound before saved games. Sound in C# is obviously another thing I know nothing about, so that will be interesting  :)

The saved game topic does raise an interesting question though. I guess I should probably be saving in the original AGI saved game format. Hadn't really thought about that until now.

I got the controllers and set.key working nicely last night. This evening I might look at a couple of bugs and then maybe at implementing the inventory screen.
I would consider to important to to have it compatible with the original AGI save game format. We would want it to work with official games.

Would trying to port the Visual AGI Sound Editor to C# help with tackling AGI sound in C#?
Title: Re: C# AGI Interpreter
Post by: Kawa on January 22, 2017, 03:16:24 PM
Code: [Select]
//From my game

public static class Vista
{
public static bool IsVista = (Environment.OSVersion.Platform == PlatformID.Win32NT && Environment.OSVersion.Version.Major >= 6);

public static readonly Guid SavedGames = new Guid("4C5C32FF-BB9D-43b0-B5B4-2D72E54EAAA4");
//You could add more, but most of them are natively supported by Environment.SpecialFolder.

public static string GetKnownFolderPath(Guid target)
{
if (!IsVista)
return null;

string ret = null;
IntPtr pPath;
try
{
if (SafeNativeMethods.SHGetKnownFolderPath(SavedGames, 0, IntPtr.Zero, out pPath) == 0)
{
ret = System.Runtime.InteropServices.Marshal.PtrToStringUni(pPath);
System.Runtime.InteropServices.Marshal.FreeCoTaskMem(pPath);
}
return ret;
}
catch (DllNotFoundException)
{
return null;
}
}
}

internal static class SafeNativeMethods
{
[DllImport("shell32.dll")]
public static extern int SHGetKnownFolderPath([MarshalAs(UnmanagedType.LPStruct)] Guid rfid, uint dwFlags, IntPtr hToken, out IntPtr pszPath);
}

Code: [Select]
//Untested -- savePath will be "Saved Games\foo", "AppData\Local\foo", or "~/.config/foo" depending on the OS.
savePath = Path.Combine(Vista.GetKnownFolderPath(Vista.SavedGames) ?? Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), gameName);
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 27, 2017, 01:58:22 PM
Thanks for the code. It will be very useful when I get to saved games.

I've now finished the inventory screen and fixed a few more bugs. There are a couple more bugs I know about that I'll take a look at this evening, and then after that it will be the menu system.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 27, 2017, 04:08:45 PM
Would trying to port the Visual AGI Sound Editor to C# help with tackling AGI sound in C#?

I'm not sure actually. I'm familiar with the AGI sound format and with playing and converting the format. So its more around the way in which sound is produced in C# that I would need to learn.
Title: Re: C# AGI Interpreter
Post by: Kawa on January 27, 2017, 04:49:21 PM
C# has no handy-dandy built-in way of producing arbitrary sound, but there's various other methods. You could lowlevel some MIDI, for example: https://code.google.com/archive/p/midi-dot-net/
Title: Re: C# AGI Interpreter
Post by: Collector on January 27, 2017, 11:00:15 PM
I was looking at that library a few years ago when I was working on a project that also dealt with MIDI. I didn't look too far into it because I decided to just use PInvoke to do what I wanted.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 29, 2017, 02:00:15 PM
Thanks, I'll take a look at that one when I move on to sound.

The menu system is now working. Still a lot of commands not implemented, but most of them aren't show stoppers for attempting to play the game. So I thought I might have a go at playing through King's Quest 1 this evening to discover some more bugs. I've already found my first two. I couldn't get in to the castle because sound hasn't been implemented yet. So I faked that by immediately setting the sound end flag when a sound is played. I then got to the throne room, bowed, and halfway through returning to an upright position, ego freezes. So I'll track that one down now, fix it, and then basically just keeping finding and fixing any bugs along the way.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 29, 2017, 04:56:59 PM
I made it up to 47 points before discovering the third bug. This one is a parsing bug. I haven't yet allowed for words that are actually multiple words, e.g. "look inside" can be (and is) defined as a single word in KQ1, in fact that are lots of cases of words with spaces in them. So looks like I'll have to take a look at my parser again...
Title: Re: C# AGI Interpreter
Post by: lance.ewing on January 29, 2017, 05:56:40 PM
Fixed that one and I've now got up to 110 out of 158 points. Unfortunately I died climbing the beanstalk. At least I've now seen that the death scene with ego flattened on the ground is working fine. Starting to miss the lack of a save game feature though.   :)

I might leave it there for now as far as the KQ1 play through goes. I'll look at implementing a couple of new features again, perhaps the looking at inventory object feature (which I don't have yet) and maybe a few other commands I haven't yet got to.
Title: Re: C# AGI Interpreter
Post by: Collector on January 29, 2017, 07:37:23 PM
Sounds like you are making good progress. This should also be good for helping flesh out the AGI library, especially the missing logic part of the library.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on February 10, 2017, 05:11:13 PM
I implemented the show.obj and show.obj.v commands over a week ago, so viewing inventory objects is working well now. Had to do a bit of after hours for work over the past week, which hindered things with the interpreter over that time. Not going to make that mistake this weekend.

I thought I'd have a go at adding the 0xFA (brush picture command) to the Picture resource this evening. As already noted, that command isn't implemented in the AGI Library, so I can't currently try playing SQ2, which is one of my favourite AGI games. SQ2 makes heavy use of that Picture command.
Title: Re: C# AGI Interpreter
Post by: Collector on February 11, 2017, 05:09:51 PM
Are you able to play through any of the official games, yet?
Title: Re: C# AGI Interpreter
Post by: lance.ewing on February 12, 2017, 04:31:47 AM
I haven't tried playing through a game again since my attempt at KQ1 the other evening. I think that one is probably achievable with luck, but I don't fancy falling off the beanstalk again until I have saved game capability built in. It's probably one of the next features I'll take a look at. Currently I'm refactoring my text window to support how SQ2 uses them in some LOGICs. For example, when Vohaul is talking to Roger, the text window is left on the screen while the animation continues. This usually doesn't happen in most games but shouldn't be difficult to support.

SQ2 has revealed a number of bugs (or missing features in some cases). I finished implementing the 0xFA brush command. The command to set the type of brush (0xF9) was already implemented, but not the brush command itself. So that's now working, and visually it appears to match the original interpreter. - The only picture I've seen so far that isn't happy in SQ2 is the title screen. The big blue letters are not filled in. So there's a fill issue, or it could be as a result of an issue in another command. It's the only picture I've seen so far that has such a fill issue. PICEDIT loads and displays it fine, so its just a matter of sifting through the code with a fine tooth comb and seeing where things differ. - I also noticed that newline characters aren't being handled in the display command. I'm surprised SQ2 was the first game to show this bug.

SQ1 hangs on the title screen, but you can get in to the game by hitting a key. My current theory about why it is hanging is that I think this LOGIC might be dependent on the sound timing. I'm setting the completion flag to true immediately for all sounds, but I suspect that this particular LOGIC relies on the sound taking a certain amount of time. I can probably fake the duration of the sound but I figure I'll just leave that until I actually have sound.

DDP has exposed some interesting bugs with regards to the synchronisation of picture placement and text. Should be easy to fix, just need to get to it.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on February 12, 2017, 05:40:40 PM
It would appear that I'll need to put a small delay between room changes:

Code: [Select]
    set(f15);
    print("You are whisked away to the airlock chamber.");
    v3 += 1;
    new.room(3);

Flag 15 tells the print command to leave the text window displayed on the screen. I'm not even seeing the window flash up, because the new.room is being actioned too quickly, probably because I've got all the resources loaded in to memory, so there's no slow disk access. Even in DOSBOX the message isn't shown for very long. I'm assuming that on the original 80s hardware it would have stayed up a little longer.

The window should stay up until the show.pic command is executed in the next room. My interpreter seems to be getting to it far too quickly. I suppose what I could do is detect that there is a window open on room change and add a small delay to account for the slower room change in the original games.
Title: Re: C# AGI Interpreter
Post by: Collector on February 12, 2017, 06:42:13 PM
Of course you can lower cycles in DOSBox to get the speed of 80s hardware. Even so, if the original was getting it from disk it is both disk and CPU clock speed.

Is it possible to set speed in the interpreter to a fixed value rather than cycles? Would this help with such issues?
Title: Re: C# AGI Interpreter
Post by: lance.ewing on February 13, 2017, 02:11:06 AM
My interpreter already has a fixed speed, by which I mean that the rate at which the interpreter cycle gets called is virtually the same as the original AGI interpreter. The interval between the start of the interpreter cycles isn't the issue but rather how fast a single interpreter cycle runs from beginning to end. This difference isn't observable most of the time, but if a particular AGI command would have taken a long time to execute in the original interpreter (let's say it might take the same time as 10 normal AGI interpreter cycles for a single command), then that's obviously a noticeable difference. My interpreter is taking no time at all for such a command.

I'd say that the text window in this particular example would need to be on screen for at least a second to give someone time to read it. So I'm assuming that whatever commands happen between that print command the the subsequent show.pic command are taking at least a second to run. This is all happening within a single interpreter cycle, because my interpretation of the original AGI specs, and of what the fragments of the original AGI interpreter source imply, is that a new.room runs LOGIC.0 again in the same cycle but for the new room. Even if I got that wrong and the new room doesn't get run until the next cycle, that would only introduce a fraction of a second delay and not enough time to even start reading it.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on February 25, 2017, 12:51:49 PM
I haven't been idle on this over the past two weeks, in fact any spare time I've had I've been continuing to slowly work towards furthering the interpreter. What's ahead of me though are a number of larger items. I'm currently looking at the save and restore functionality. I think what I'll end up doing is to support the save game format in use between versions 2.4XX up to 2.936. The interpreter doesn't yet load AGI v3 games, so I haven't looked at the saved games for those games. What I've noticed for AGI v2 though is that the 2.272 saved game format differs in a few places. It has less space for strings, less space for key mappings, and appears to have different fields toward the start of the first "piece", which the original interpreter calls the "savevars" piece. I might be able to support loading saved games from that version at some point, but I'll focus on versions from 2.4XX for now. The only difference I can spot at the moment between 2.4XX saved games and the the 2.9XX saved games is the addition of a field for storing the pushed script number for the push.script command.

My interpreter doesn't currently support the concept of script events, so in order to support saved games, I'm going to have to implement that as well. There's a whole section of the saved game format dedicated to script events. It's important for things such as restoring add.to.pics.

On the horizon for some point (perhaps towards the second half of March) would be sound implementation. Then maybe AGI v3 support in April.
Title: Re: C# AGI Interpreter
Post by: Collector on February 25, 2017, 07:25:09 PM
It would be great to have sound support added to the library.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 05, 2017, 03:40:31 PM
In working on the AGI interpreter saved game functionality, using the original saved game format, I've noticed that it doesn't appear to have been documented yet, at least I couldn't find it anywhere, other than a few notes by David Symonds on the scummvm web site that was only scratching the surface. There is enough information in the original AGI source code fragments to piece the whole thing together. We probably should document it somewhere.
Title: Re: C# AGI Interpreter
Post by: OmerMor on March 06, 2017, 02:51:30 AM
There is enough information in the original AGI source code fragments to piece the whole thing together. We probably should document it somewhere.

Good idea.
Why not the ScummVM wiki?
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 06, 2017, 07:51:06 AM
Yeah, that was my thought. I've emailed David Symonds and he is quite happy for me to do that. He did have a note on there asking people to let him know if they were going to make changes to the page, but he says he's going to remove that note now. He hasn't touched AGI for 10 years.
Title: Re: C# AGI Interpreter
Post by: Collector on March 06, 2017, 11:41:28 AM
Be sure to add it to the AGI Wiki as well.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 06, 2017, 02:38:05 PM
It appears that I don't have an account for either wiki at the moment. I'll email sev for the scummvm one. Can you check the AGI wiki one? I thought I had an account, in fact I have the email with the details from back in 2015, but it's saying an account by that name doesn't exist. I probably let it expire.

Edit: I've got the scummvm wiki account now, thanks to sev. Probably won't have a chance to start adding anything until mid-week, although I might surprise myself.
Title: Re: C# AGI Interpreter
Post by: Collector on March 06, 2017, 04:44:59 PM
Yes, you do have an account (http://agiwiki.sierrahelp.com/index.php?title=User:Lance_Ewing). Perhaps you forgot your password?
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 07, 2017, 01:39:52 PM
It seems I forgot my password and my user name. All working now. Thanks.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 07, 2017, 02:03:34 PM
I'm mentally scoping out the Saved Game format page. I've noticed that David Symonds slot a link to the page in between sections 9 and 10 on the scummvm wiki:

http://wiki.scummvm.org/index.php/AGI/Specifications#Savegame_files

I think I'm going to move that as part of the work. I guess it could become a section numbered 14 for the purposes of the scummvm AGI specs.

It would be nice to keep the AGI Wiki in sync. How do you feel about incorporating sections 12 and 13 from the scummvm wiki version of the AGI specs in to your wiki and then I can put Saved Games as section 14 on both wikis? I'm in two minds about it.

I was also debating whether Saved Games lived under one of the existing sections rather than as its own section but couldn't decide what section was most appropriate. Perhaps 11.4 under Other Information?
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 08, 2017, 03:14:11 PM
Be sure to add it to the AGI Wiki as well.

I notice that you've been updating the AGI specs on the AGI Wiki and have introduced a copy of the Savegame Files page from the scummvm wiki. I'll make a start tonight on filling that page out with the full details of the format.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 08, 2017, 06:50:28 PM
I think I've done everything except for the Animated Objects section. That one is going to take a while and it's time to rest for this evening. Progress is here:

http://agiwiki.sierrahelp.com/index.php?title=AGI_Specifications:_Chapter_10_-_Savegame_Files
Title: Re: C# AGI Interpreter
Post by: Collector on March 08, 2017, 11:35:56 PM
Great! I noticed that the savegame entry on the ScummVM Wiki was little more than a stub that was stuck into the old AGI documentation without adjusting the chapter numbering, so I added the stub for you to flesh out and integrated it into the documentation better.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 10, 2017, 06:13:47 PM
I've completed the Animated Objects section of the saved game format page this evening. I probably need to proof read it a couple more times, and then add another section at the bottom explaining a few things that are done by the interpreter as part of the restoring of a saved game file. When I've got it to what I think is a finished state, I'll copy it over to the scummvm wiki.
Title: Re: C# AGI Interpreter
Post by: AGKorson on March 12, 2017, 07:06:29 PM
Lance, have you looked at Nick Sonneveld's NAGI source files? NAGI was probably the best modern interpreter ever written, and Nick's source code was really easy to follow. There might be a lot of stuff you could borrow from there to finish your C# interpreter faster.

I used his source code a lot while working on WinAGI, and I also tinkered around with it, adding support for changing color pallets and extended fonts.

He used a resource called SDL for graphics and sound, so there might be some lower level functions that won't necessarily be useful. But if you haven't looked at it, it just might make your job easier.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 12, 2017, 07:33:23 PM
No, I haven't had a look at the NAGI source code yet. I've spent a long time looking at the fragments of original AGI interpreter source code though. It is really amazing that so much of the original interpreter code was found in the unused parts of a game disk. Usually what you find in the slack space on those disks is quite fragmented, but these are whole files and quite a lot of them as well. Apparently it came from an original 720K SQ2 v2.0D game disk. If you haven't yet had a look at that source, there's a post in these forums with a zip attached containing all the source that was extracted.

I've already implemented quite a high percentage of the interpreter. I don't think my rendering is 100% accurate though for some of the non-standard features that are possible. Over the past couple of weeks I've been trying all sorts of things to see what is possible, things that scummvm crashes with but the original interpreter will quite happily do. My interpreter doesn't handle them either. Haven't tried NAGI, but that would be interesting as a comparison.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 18, 2017, 06:04:41 AM
I've been thinking that I should probably call this new C# AGI interpreter something different than MEKA. Originally the idea started out as being a conversion of my old C interpreter code to C#, but that's not really how it panned out. I've ended up building it from scratch pretty much but using about four different sources for the information. I did, however, call the project MEKA within the repo. So I'm now considering changing that to something else. The original M.E.K.A, although built solely by me, used the initials of the surnames of Joakim Möller, Peter Kelly, and myself. I did that as an acknowledgement to their contribution back in the early days when it was pretty much just the three of us that split up the work of building tools and creating the AGI specs, etc. It doesn't seem appropriate for the new interpreter though. So open to some ideas about what to call it.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 22, 2017, 05:56:58 PM
AGILE it is then.
Title: Re: C# AGI Interpreter
Post by: MusicallyInspired on March 22, 2017, 06:24:53 PM
Hey that's great! I like that. It's perfect. I couldn't think of anything.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 26, 2017, 11:41:32 AM
I believe I've got the saved game "restore" functionality working in a compatible way to the original games (assuming they're between 2.4XX and 2.9XX). I actually got that working before I began working on the "save" function, which I realise seems the wrong way around. The way I was testing it was to save games with the original KQ1 and then loading them in my interpreter. There are probably a few edge case bugs in there somewhere that will come out at some point, but the core restore state seems to be working.

The "save" function is coming along. I've done the first two sections, i.e. the general state vars and the animated objects. I'm now working through the inventory objects, which is leading me in to implementing the Encode method of the Objects Resource in the AGI Library. That should therefore be reusable for the IDE. It has to support an Encode in plain form and encrypted form since the saved games store it without the Avis Durgan XOR but most games store the OBJECT file with the Avis Durgan XOR. The third section of the saved game file, i.e inventory items, is basically the unencrypted OBJECT file without the 3 byte header. So for the third section, I'll just ask the AGI Library for the unencrypted form and then write out everything after the first 3 bytes.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 30, 2017, 04:10:44 PM
I have the "save" functionality working now and it too appears to be compatible with the original interpreter. Haven't done much testing yet, but I was able to load a game saved in my interpreter in to the original interpreter. No doubt there will be some broken bits here and there that I'll discover when fuller testing is done.

I'm now looking at the simple save functionality. I think I already have that functionality in place, but when I tried it with Mixed Up Mother Goose with a clean slate, i.e. no pre-existing saved games, it asked me to pick my name from a list that was empty, and the pointer was able to move up and down over the whole screen rather than just within the empty list.

Turns out that the original game did this as well, which I found quite strange. I'm using 2.915 of the interpreter. Can others that have the AGI MUMG give that a go as well, i.e. try firing up the original AGI interpreter with MUMG and no saved games and confirm that there is a bug there with that initial name selection dialog?
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 30, 2017, 04:25:20 PM
That's a little sad actually (and exciting as well; a strange feeling really). Turns out this isn't the first AGI interpreter written in C#. I was trying to discover any mention of the MUMG bug mentioned above and happened to find this web site:

http://www.huguesvalois.com/Resume.aspx

where it says this:

Quote
Developed an interpreter for Sierra AGI games in C# by porting an existing C interpreter (NAGI). Added a significant number of features not found in the NAGI. Used SDL for graphics, input and sound.

and this:

Quote
Developed a suite of tools in C# for creating Sierra AGI games. This includes compilers, decompilers and editors for the various types of resources present in AGI games. Based on existing AGI specification documents describing the resource file formats.

Was anyone aware of Hugues Valois and his efforts?

Edit:

Check out this write up: http://www.huguesvalois.com/ProjectsAgiPlayer.aspx

Some very interesting features. Haven't actually found a link to it yet.
Title: Re: C# AGI Interpreter
Post by: lskovlun on March 30, 2017, 04:46:08 PM
Was anyone aware of Hugues Valois and his efforts?
The name sounds familiar... but I can't remember whether it was in a FreeSCI context or on some forum.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 30, 2017, 04:51:34 PM
A search for FreeSCI and his name does turn up a number of results, e.g. https://lists.gnu.org/archive/html/freesci-develop/2006-01/msg00005.html

and in this source file:  https://github.com/RangelReale/freesci-pnd/blob/master/src/tools/musicplayer.c

Edit:

He did the game selection menu in FreeSCI as well:  http://scummvm.sourceforge.net/credits/
Title: Re: C# AGI Interpreter
Post by: Collector on March 31, 2017, 11:00:58 AM
Interesting. The copyright on that page is only a couple of years old, so perhaps he can be contacted.

Anyway, my MMG has the same behavior. It does,however have the same interpreter as yours.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on March 31, 2017, 01:06:28 PM
I've sent him an email. Looks really good, doesn't it?

Regarding MUMG, I guess we could try swapping in different interpreter versions and see what we get. Would have to be in the 2.9XX range, and probably greater than 2.915. So that probably leaves 2.917 and 2.936 to try. Would be interesting to see if the same behaviour occurs in AGI v3 as well. For my interpreter, I'm not sure whether to leave the bug in there, or whether to skip that dialog, which is pretty much what you're forced to do in that state, i.e. hit ESC and move on to the next screen where you enter your name.
Title: Re: C# AGI Interpreter
Post by: Collector on March 31, 2017, 06:31:45 PM
His interpreter was using SDL like NAGI. Were you using SDL for your C# interpreter? If not it still might be worth it to continue your new interpreter. Even so, it would be nice to see his code.

A quick test with 2.917 and 2.936 dumps you back to the C: prompt and a "Bad Image File" message respectively.
Title: Re: C# AGI Interpreter
Post by: lance.ewing on April 01, 2017, 05:05:30 AM
What I did to try out 2.936 was to find one of the fan games that uses that version (of which there are quite a lot) and copy the interpreter files from there. I think they use cracked versions, but that doesn't really matter for the purposes of this MUMG test.

Having done this, I can confirm that 2.936 also has this bug with that simple name selection dialog.

Regarding the question about SDL, no, I'm not using SDL.
Title: Re: C# AGI Interpreter
Post by: Collector on April 01, 2017, 11:16:17 AM
I didn't think that you were using SDL.