Author Topic: AGI reverse engineering with AI  (Read 165 times)

0 Members and 1 Guest are viewing this topic.

Offline pmkelly

AGI reverse engineering with AI
« on: Today at 01:24:14 AM »
Hey all,

Like everyone else, I've been experimenting a lot with AI coding assistants lately (specifically Codex), in particular trying to explore their limits on difficult tasks. After some promising results seen with reverse engineering C64 binaries, I thought I'd have a go at doing the same for AGI. Everything in the docs linked below has been generated from scratch, given only the executable files and game data of AGI games, without consulting any existing documentation or implementations (SCUMMVM, AGILE etc). My goal was to see what it could do given a completely unknown binary.

While none of the work it's produced is novel (this is all well-trodden ground thanks to the work of the community), I've been able to get it to generate an *extremely* detailed analysis of every aspect of the interpreter and produce a high-level spec, including coverage of different subsystems and different interpreter versions ("profiles"). There's only a few outstanding details left but I think it's pretty close now.

I'm currently having it generate playthroughs of games based on analysis of the logic scripts and use of the QEMU debugger to interact with the interpreter. Once I have winning playthroughs for all games, I'll use that as a corpus for validation and see how well I can get codex to produce a new implementation of the interpreter in Rust.

Spec: https://peterkelly.github.io/agi-re/spec/

RE documentation: https://peterkelly.github.io/agi-re/docs/

Github repo: https://github.com/peterkelly/agi-re



Offline cosmicr

Re: AGI reverse engineering with AI
« Reply #1 on: Today at 01:47:17 AM »
Very cool - it has created a great resource. I have done some similar experiments with both AGI and SCI. I am keen to see what it's new implementation in Rust will be like. One of the things I found is that it had some novel ways to manage memory.

I feel this kind of reverse engineering by AI could really accelerate a lot of retro gaming work. I could foresee a lot more classic game engines added to SCUMMVM for instance. And ports of engines to systems that people wouldn't have otherwise considered. Not just adventure games either. Look at existing projects like C&C or Jagged Alliance, Mario and Sonic, etc.  I bet if you tried hard enough you could port AGI to the Commodore 64, perhaps with some limitations (eg using a REU, and other tricks), of course with the assistance of an AI model like Fable etc.


Offline lance.ewing

Re: AGI reverse engineering with AI
« Reply #2 on: Today at 04:49:58 AM »
While none of the work it's produced is novel (this is all well-trodden ground thanks to the work of the community), I've been able to get it to generate an *extremely* detailed analysis of every aspect of the interpreter and produce a high-level spec, including coverage of different subsystems and different interpreter versions ("profiles"). There's only a few outstanding details left but I think it's pretty close now.

I'm currently having it generate playthroughs of games based on analysis of the logic scripts and use of the QEMU debugger to interact with the interpreter. Once I have winning playthroughs for all games, I'll use that as a corpus for validation and see how well I can get codex to produce a new implementation of the interpreter in Rust.

Spec: https://peterkelly.github.io/agi-re/spec/

RE documentation: https://peterkelly.github.io/agi-re/docs/

Github repo: https://github.com/peterkelly/agi-re

Wow! I'm going to be taking a closer look at this later today.

You mention that none of it is particularly novel. One area that isn't well understood is the evolution of the interpreters before AGI v2, i.e. the GAL interpreter and the AGI V1 interpreter versions. It would be interesting what you get if you give it the disk images for those original game disks, i.e. the KQ GAL disks (for IBM PC JR and IBM PC), then the KQ2 and BC disks that use AGI V1. This would be a great way that AI could make a big contribution to the understanding of the early history of AGI.

I actually did something a few months back with AI and AGI. I asked it to reverse engineer the Apple IIe/IIc AGI interpreter used on the Gold Rush game, which would be probably the last version of the AGI interpreter for the Apple II computers. It produced a fairly decent 6502 disassembly, completely documented, and some supporting markdown documentation of how it all works. I will put that in a repo and point people to it soon.

I started out by asking AI to convert AGKorsson's WinAGI Windows help file to Markdown first, so I had an easily understandable (by AI) set of documentation on how AGI works. WinAGI's help file is easily the best and most complete AGI documentation resource out there. I gave it that markdown output, and the original AGI source code for the IBM PC version, and it used that to help with its understanding of things like terminology and what it should expect to see in the 6502 disassembly.

Offline lance.ewing

Re: AGI reverse engineering with AI
« Reply #3 on: Today at 04:57:21 AM »
I bet if you tried hard enough you could port AGI to the Commodore 64, perhaps with some limitations (eg using a REU, and other tricks), of course with the assistance of an AI model like Fable etc.

I also think it should be possible these days to port AGI to the C64 (someone has actually almost finished a C64 port of Leisure Suit Larry already, for the C64, but from what has been said about it, it doesn't sound like an AGI interpreter as such). - As you say, it would definitely need a REU.

Someone had an AGI interpreter for the BBC Micro working a few years back, another 6502 based computer.

Offline pmkelly

Re: AGI reverse engineering with AI
« Reply #4 on: Today at 05:18:25 AM »
I bought a C64 Ultimate as soon as they were available and it's a great machine and I want to explore it a lot more. You can drive up the CPU speed and it also has a built-in REU, both of which may be necessary.

The main challenge in getting AGI onto the C64 is graphics. From what I understand, both the bitmap modes and the character/tile modes are insufficient to accurately display a full EGA picture, due to limitations such as the number of allowable colours. Additionally, the C64 is really designed more for tile-based graphics (40x25 I think).

I saw an interview with Ron Gilbert once discussing the development of the C64 version of maniac mansion. Apparently when creating the artwork they started out with a detailed image and then had to reduce it down to something that could be expressed in at most 256 tiles. I reckon the most promising approach is to not have the engine actually run all the drawing commands in-game, but rather to have a conversion tool which pre-renders and picture into tiles, and then builds disk image(s) containing the converted resources. The priority screen will be tricky though.

I don't know anything about how the Apple II interpreter worked (i know this was used for KQ at least). It might be easier to try to port that to the C64 instead. Either one would be a fun challenge.

Offline lance.ewing

Re: AGI reverse engineering with AI
« Reply #5 on: Today at 06:04:49 AM »
The main challenge in getting AGI onto the C64 is graphics. From what I understand, both the bitmap modes and the character/tile modes are insufficient to accurately display a full EGA picture, due to limitations such as the number of allowable colours. Additionally, the C64 is really designed more for tile-based graphics (40x25 I think).

To be honest, I'm not that familiar with the C64 either. I'm very familiar with the VIC 20 but that's definitely not an option. The graphics on the VIC 20 are far too limited. On the C64 though, I don't know this as absolute fact, because AI told me this, but apparently three of the colours can be different every 8x8 character? - I asked AI what it thought about the graphics limitations on the C64 in relation to AGI. It suggested that AGI pictures in general don't use a lot of colours per 8x8 block, so maybe in most cases it might look okay. - But there would certainly be some cases where it would need to use a different colour in certain pixels due to the limitations. I compare that with the Apple II AGI graphics, that also look a bit less than perfect, with various colour artifacts clearly visible. The C64 might be able to use raster effects, or maybe some stationary sprites to fill in bits that don't otherwise look good. Some of the games that have come out on these old retro computers these days are quite amazing, so there are bound to be tricks used by those devs that could be applied.

I reckon the most promising approach is to not have the engine actually run all the drawing commands in-game, but rather to have a conversion tool which pre-renders and picture into tiles, and then builds disk image(s) containing the converted resources.

Might require a lot of disk space, if disks are your target. I think the C64, even with the modern SD card options, would still be restricted by available floppy disk space for the disk images, so you'd be switching "disks" a lot. One of the annoying things about the Apple II AGI games is having to swap the disks all the time. I don't have a physical Apple II machine, only an emulator, but it is rather annoying. - I think a C64 version of AGI might need an alternative way to load data, like a cartridge with lots of storage space that it can very quickly load from, rather than from disk. Maybe with a REU, something can load all the data into RAM up front, so that it becomes really quick changing rooms.

I don't know anything about how the Apple II interpreter worked (i know this was used for KQ at least). It might be easier to try to port that to the C64 instead. Either one would be a fun challenge.

The only AGI game that didn't come out on the Apple II computers was Manhunter: San Francisco; all the rest did. I guess by the time that game came out in 1989, they didn't think it was worth releasing an Apple II version, although they did release Gold Rush, which is interesting. Someone could probably take the interpreter from the Gold Rush game and the data files from the IBM PC version of Manhunter: San Francisco and create an Apple II version of the game.

Regarding porting the 6502 disassembly to the C64, it is funny that you say that. I did set out trying to do that after creating the Apple II disassembly, but ended up getting completely stuck in some kind of loop with the AI that spanned over several days. Burnt through a lot of AI credits and it kept going around and around in circles on one particular thing. And with the code all being written by AI, I didn't feel like I could take over. I'd be quite happy to pass it over if you like, in case any of it is useful. It might even be a good approach, for someone else to take over, get their AI to analyse what is there at the moment, identify the gaps, look for existing bugs, etc.

Offline lance.ewing

Re: AGI reverse engineering with AI
« Reply #6 on: Today at 07:25:20 AM »
I have tidied up the Apple IIe/IIc AGI interpreter disassembly a bit and put it here:

https://github.com/lanceewing/agi-apple-ii



SMF 2.0.19 | SMF © 2021, Simple Machines
Simple Audio Video Embedder

Page created in 0.028 seconds with 16 queries.