Leaked Original SCI Source

[Kawa]

I understand the work it would take to do a new template game using the SCI script

Do you want a Studio-to-Script converter? Cos I've been looking for a parser challenge.

I may or may not be joking.

[troflip]

Well, you could assume from the GK decompiled code that this functionality already exists (not to mention the github comments I hinted at before).

The compiler already works with a syntax tree. The steps are:
Studio source code -> (parser)-> syntax tree -> (compiler) -> byte code

I've written the Sierra script parser, so it was mostly a matter of just substituting the new parser:

Sierra source code -> (new Sierra syntax parser)-> syntax tree -> (compiler) -> byte code

The parser wasn't exactly trivial, and it's probably not easy code to understand if you didn't write it, but it's not too bad. I can basically write the grammar in BNF form using C++ operators, very similar to Boost Spirit. Only took a 3 days or so.  (Separate from that, minor modifications to the compiler were required to support continue statements, and multi-level break/continue).

And the decompiler does the opposite:
byte code -> (ridiculously complex decompiler) -> syntax tree -> (source code formatter) -> source code.

And clearly from GK1 source provided I've already got the Sierra source code formatter working:

byte code -> (ridiculously complex decompiler) -> syntax tree -> (Sierra source code formatter) -> Sierra source code.


From the above, you may deduce that the pieces are in place for the following:
Studio source code -> (Studio parser) -> syntax tree ->  (Sierra source code formatter) -> Sierra source

And you'd be correct!

You'd also be correct if you assumed I have implemented one-click functionality to convert an entire project from Studio syntax to Sierra syntax, and that I did this with the template game and it all successfully compiles and runs (to the best of my knowledge) without problems. I'm sure there are some bugs somewhere, but I haven't found any major ones yet.

You could also technically go back to Studio script from Sierra script, though I haven't tested that yet, and I know it will fail for some cases (like continue statements, which have no equivalent). So let's call it "not supported".

The benefits of having the syntax tree abstraction in there are huge. It made things like the Insert Object/Insert Method script editor functionality nearly "just work". Because that was never directly pasting source code... it parsed source code (the stuff in the Objects subfolder of SCI Companion) into a syntax tree, made the necessary modifications, and then spit it out using the source code formatter, and pasted that.

I've tried to make the Sierra syntax code formatter much better than the Studio syntax code formatter. So if you see badly placed parentheses or badly-indented stuff, let me know. I haven't even looked at the GK1 source code I provided. I've just been looking at the template game.

The major thing left to do is to fix up the autocomplete functionality for the new syntax. And then some polish and what not...

[troflip]

I should also mention I've taken some deviations from Sierra script syntax, much of it due to the way the compiler is set up (and requiring compatibility with SCI Studio). It's possible I can make it closer to the original, but only if I don't need to maintain two separate ways of compiling. The more code I can re-use the easier it makes testing, etc...

Anyway, some differences:
- : or ? will be required for selector names. Sierra's code appears to have stuck with this convention anyway, so not a big deal.
- , will be optional (instead of required) when separating selector calls (made possible by requiring : and ? above)
- procedures and methods don't need to be forward declared
- Currently I'm still requiring the "use" statements that Studio syntax uses ( ). Sierra didn't have this. As far as I can tell, all classes were in global scope (so they could always be referred to). And procedures required an "extern" statement that mapped procedure names to script/export index pairs, and which was typically included in a header file I think. I'm tempted to try to replicate this, but it might mean maintaining two different code paths for part of the compilation process, and I'm a bit reluctant to do that.

Some possibly irritating things about the Sierra syntax that I *won't* change:

- all operators have to have white space after them:
    (++i) -> this is a compile error
    (++ i) -> this is valid

- it is *very* strict about parentheses. Unlike Studio syntax, you cannot arbitrarily enclose expressions in parentheses. Luckily this makes it much easier to parse than Studio syntax.
    (if (something) 1 else 2)  -> compile error, because something is enclosed in parentheses. We think it's a procedure.
    (if something 1 else 2) -> valid
    (if (== something TRUE) 1 else 2)  -> valid, since all expressions have to be enclosed in parentheses
    = temp0 6  -> compile error
    (= temp0 6)  -> valid!
- the array syntax is unorthodox, but I'm not changing that


The Sierra syntax overall is easier to parse than Studio syntax. It's more logical and consistent (and compiles are about 20% faster).

[Collector]

Part of my curiosity is from going through the docs and noting the differences. The way my brain is wired the Sierra syntax seemed to be easier to understand.

BTW, the Kernel Functions document is fully Wikified, including the index.

[OmerMor]

troflip,
your work is outstanding.
I'm sorry my leaked source gave you so much work. I wish I had it sooner, so you won't have to do stuff twice.

Thank you for doing so much for our small community!

[OmerMor]

Did you improve the decompiler to support more constructs?
Is it able to decompile stuff that used to fallback to disassembly?

[troflip]

Nope, I've done no work on the decompiler. So, that means you'll never see continue or contif in any of the decompiled source code (and those will probably fall back to assembly). Honestly, the decompiler was a real headache to work on and test, so I'd like to leave it as is.

The other "new" statements, like cond, repeat or breakif will, however, show up in decompiled code. Those are just transformations of already-existing statements like ifs and loops.

[MusicallyInspired]

Indeed, troflip. Thank you so very much. I really need to get back to a SCI.1 project, too. Without you Collector, Gumby (sometimes), and I would still be here trying to make sense out of the SCI Studio 4 source code which is beyond any of our capabilities. I was still trying to figure out how to get SCI1/SCI1.1 pictures to display. I acquired an ancient version of Borland just to open the SCI Studio project and get it to compile. Half the stuff was never completed and things kept causing crashes. I fixed simple bugs, but I couldn't do anything else. The bitmap import function was incredibly gimped because what I did get working came out all distorted and the wrong colours. So thank you!!

Here's an off-topic question, the VGA SCI1 template game is just sitting there in the SCI Studio 4 source. What are the chances of getting that working? Did SCI Studio 4 VGA also have a compiler for SCI1 included? I'm not asking anybody to work on it, I'm just curious how close Brian actually was.

[Kawa]

Speaking of Brian's work, does anybody have a copy of that VGA demo he'd released?

[OmerMor]

Speaking of Brian's work, does anybody have a copy of that VGA demo he'd released?

Here you go.

[Kawa]

Thank you.

Is it me or are those picture resources so messed up only SSCI can load them?

[Collector]

SV does not.

[Kawa]

Not to mention the LOOKUP_ERRORs everywhere in the decompilation -- I've never seen those before.

[troflip]

That demo is actually pretty cool, lol.

"SCI Studio VGA will be better than SCI Studio EGA!"

Looks like he was using the interpreter from Leisure Suit Larry VGA (1.000.577). Companion completely mis-recognizes the resources as being from SCI0 EGA. So does SV.exe, I guess.

btw, if you open the game version dialog and switch the pic format to VGA 1 and click reload, Companion will show the pics correctly (you have to do this each time you open the game, I should really make it stick).

Anyone want to make a SCI Companion VGA game demo?

[Kawa]

I read that as "redo this for Companion" and I was like SURE