Can SCI script be modernized into the current age of programming languages?

[deckarep]

Hey SCI Programming friends,

I've long dreamed of getting access to SCI scripts in a modern programming language. For a myriad of reasons really. I know the games can currently be manipulated via tools like SCICompanion and other open-source tools, and even some original tooling that got released over the years.

But even though I have mad respect for all the developers out there who have reverse engineered the Sierra games and built such tools. And even though I have mad respect for the original Sierra devs including the likes of Jeff Stephenson for creating the original SCI engine and kernel...what I don't like is reading an old Small-talk, Lisp style language with a bajillion parenthesis and prefix-notation for expressions. It makes my eyes bleed.

So what if SCI script was taken out of the stone-age and the games were modernized in such a way that the source code becomes easily readable/writable and usable with modern tooling and modern software conventions?

See this Github gist for more details including my rationale: https://gist.github.com/deckarep/a4af4b8b16851e65688a94f445fb1a8e

I welcome any comments and criticism of this effort. I have no illusions: it will be a hard task to get to a point where a game can run and I probably won't take it that far at least by myself.

The transpiler is not yet open-source...it's in an ugly state and I at least want to ensure it runs on a full game to completion.

Thanks,

-deckarep

[Kawa]

It means, the original VM intepreter and garbage collector goes away, Python has its own

I feel like it wouldn't be SCI any more, then.

Now, there was an early attempt to supplement SCI Script (both flavors) with a more C-style variant that, unlike the above quote, would still compile to the same bytecode. I couldn't tell you much about it -- I can't even recall if it was Brian or Phil who tried.

Based on a combination of Small-Talk and Lisp, with Prefix notation for expressions...it can get unwieldy to read at least for me. Boat-load of paranthesis and expressions that don't read like normal infix notation.

They're prefix notation specifically because it's Lisp-like. Now if you'll excuse me, I have to barricade this door so the ravenous Clojure fans don't get in. For your safety.

Speaking of Clojure, according to this survey from 2023 differences in common idioms were the largest stumbling block among those same fans. So don't worry, it's not just you... for long.

(Did you know the Sierra SCI toolkit includes a CG-to-SC transpiler? I ran it on a Larry 1 room for a laugh. Never thought I'd see Technically Valid SCI code that reads like AGI.)

[lskovlun]

Now, there was an early attempt to supplement SCI Script (both flavors) with a more C-style variant that, unlike the above quote, would still compile to the same bytecode. I couldn't tell you much about it -- I can't even recall if it was Brian or Phil who tried.

There was an even earlier attempt by Dark Minister. As far as I know he only decompiled one script by hand, SQ3's script 0 (attached - I could probably find it in the archives, but this is easier).

But I agree with Kawa. Game constructs can be expressed naturally in SCI in part because of its language features (easy setting of properties, cond statements...). Take the language away and you might as well use a different game engine too.

[deckarep]

Thanks for the replies!

It's quite interesting to see that file posted by @lskovlun where it looks like somebody previously tried to do the exact same thing.

One thing I did want to make clear is that I 100% agree that going in a direction like this starts to make the code a lot less "SCI" which also means the new code loses its charm and its essence to one of the things that makes SCI and Sierra great.

But also consider, that the source code and the games themselves are very much a part of what makes Sierra and the SCI engine unique while IMO the current representation of the source is just a means to an end and something that is still very much "locked up" in tooling that was grounded in technology of the late 80's and early 90's.

In terms of expressiveness, so far the only thing that Python doesn't have is a switch statement but that is easily replaced with the new match statement. A traditional for loop would need to be replaced with a while loop. And lastly, cascade syntax isn't supported but many, many languages don't support that. Dart by Google does, and I've considered that language too.

This attempt is more at unlocking the source and engine for modern times vs preserving what once was. Actually, SCI's legacy is already preserved through these forums, ScummVM, SCICompanion and all the existing tools whether proprietary or not. So much great progress has been made in the spirit of preserving the exact representation of the games.

What I'm thinking is more along the lines of what the existing engine and programming language looks like when modernized.

Also, if there existed a 100% SCI transpiler to some other language like Python, it doesn't prevent anyone from continuing to work in the SCI expressed language and having the transpiler be a pre-compilation step to the build process. At that point, the transpiler just becomes a back-end implementation detail.

Anyhow, I do appreciate the replies and thanks for the insights that you both provided. This has been a fun project and if anything has provided me with a learning opportunity and an opportunity to better understand what makes these games special.

-deckarep

[Kawa]

Compare and contrast all this with the idea I had back in October involving a custom interpreter that's stripped down to run in text mode and replaces the event system with a "get line of input" kernel call. That'd still run PMachine and could be targeted by SCI Companion and SC alike.

[deckarep]

Interesting,

I've also wanted a stripped down PMachine compiler but for a different reason which is simply to evaluate raw code for testing purposes.

Was your idea to have a minimal PMachine for text-based adventure games? I suppose it wouldn't be too hard to pull this off and might be a fun one actually.

I actually think the SCI PMachine was quite, quite sophisticated for its time. Especially going the object-oriented route when around that time "object-oriented coding" was still largely new and unheard of.

[Kawa]

Was your idea to have a minimal PMachine for text-based adventure games? I suppose it wouldn't be too hard to pull this off and might be a fun one actually.

Would've been more fun if they hadn't gutted the actual parser X3

Cos all I got is SCI 1.001.100, so no parser there.

[Charles]

It sounds like a neat personal hobby project, but I'm a little unclear on your actual goals here...

This attempt is more at unlocking the source and engine for modern times vs preserving what once was.

What I'm thinking is more along the lines of what the existing engine and programming language looks like when modernized.

I can understand modernizing the source, but you've lost me at modernizing the engine.  Kinda like what kawa said, if you're making drastic "modern" changes to the engine maybe you're better off just using a different engine?  Like AGS, or even Unreal or something.

Making changes to the source-code to be more python-like or more java-like, or whatever is one thing, and that's cool... may open up the SCI engine to a wider user-set, but you wouldn't want to convert sci-script to actual python, would you? Like you already have a specific byte-code target that your code should compile to (i.e. the SCI engine, be it original DOC interpretors or ScummVM interpretor), and it'll never be python or java or c#.

But that doesn't mean you can't write a compiler that takes c#-like, or python-like code and compiles it to SCI byte-code.  That still has to be your ultimate target, right?

[deckarep]

Hi @Charles,

Well my primary goal is to transpile an existing game to a fully different language. The main goal there is honestly just readability because I do suffer from dyslexia, so reading expressions in prefix notation riddled with a lot of parenthesis is difficult for me personally. This is all born out of curiosity to just study the game logic of the classic Sierra games as-is.

The bigger question though: could such code get to a point where it was runnable? At first it would be first simple small scripts. The kernel code would need to interop with this new code, for a language such as Python: it's already run on a bytecode interpreter. Aside from a handful of SCI runtime concerns + kernel routines, the Python VM would be just as powerful if not more so.

So while I could take the converted Python (or other target language) and now compile that to true SCI bytecode, it's still going back into an older proprietary format IMO.

When I talk about modernizing the engine, I don't necessarily mean targeting newer engines like Unreal or Unity to have access to 3D and new asset management. I simply mean interoping with all the kernel routines, creating a window and rendering the relevant assets: pics/views/polygons/priority screens, etc from Python and that would be "good enough"

From there you can go further though. What's stopping someone from importing a Python library that does some HTTP requests, or a TCP socket and now you are unlocking something that could be multi-player. Of course, the original game's aren't going to just suddenly become network aware without changes as well. But the real idea is that modern tooling and libraries are now immediately available by moving to entirely different VM implementation. So the dream goal although quite hard would be to migrate SCI off of the original PMachine VM and language and moving it into modern language and runtime while being able to run all games as-is.  If I did my job correctly, the games would all play with no discernible difference. That would be a litmus test into a full proper migration.

Once you are there, you can crack open a game like King's Quest, import a JSON library, import the HTTP requests module and now dynamically load in object inventory from some remote server and now you have power to do some really neat stuff that was previously not possible.  Another idea is to dynamically send the games text to a 3rd party text-to-speech API and have a character speak in real-time even if the game was never made as a "talkie" variant. Another idea is to have an NPC wired up to Chat-GPT and now it can tell about the magical adventures they experienced on the Land of Green Isles.

"Now you're just talking non-sense deckarep!"

I'm sure my ideas sound like I'm smokin' crack...

For what its worth, I'm sure people would be like: Um no, please leave these classics alone. But I see it differently, I'm more in the camp of: Bring adventure games back into the modern era and let's see what they can do!

=)

-deckarep

[Charles]

Sounds like a fun project. Best of luck to you. 

[Kawa]

I'd like to meet your dealer, Deck.

[deckarep]

Thanks @Charles!

Haha, @kawa well said

[lskovlun]

I actually experimented with a script-driven TCP connection handler many years ago. Written in C++, but with a script class modeled on SCI.

[deckarep]

Oh that's cool. What was your use-case or intention with doing this?

I know one thing to be aware of is ensuring that any IO-blocked call doesn't block the main game loop. But that is easily dealt with by using the Python async_io framework in my case or just ensuring that all remote calls are ran in an external thread and dispatched back into the main game loop thread when the result is ready.

I think it opens up use cases: perhaps a Realm like experience with some work of course.

-deckarep

[lskovlun]

I had no other intention than trying it.  And the nice thing is, you don't have to block. You can use signal-based I/O, as I did. When you get an I/O completion signal, you either delegate to (cue) the appropriate script right away, or put that script on a delayed cue list as in SCI. You initiate reads or writes like you would a Motion object in SCI.