SCI Companion V3 - alpha build notes/bugs/feature requests

[Charles]

I have v1.200.

When I decompile it from scratch, I get the same as yours (except with text resource 0 64).

Code: [Select]

(procedure (proc0_27)
(= global4
(proc255_0
0
64
80
{Giving up, huh?}
81
{Quit}
1
81
{Don't Quit}
0
82
800
1
4
)
)
)

If I then rename proc255_0 to Print and re-decompile, it comes out as

Code: [Select]

(procedure (proc0_27)
(= global4 (Print {How about a slice of quiche?}))
)

Is Print a reserved word?  I don't see it under the list of Kernel commands.

[troflip]

There's nothing special about the Print procedure in general, but it is handled specially by the decompiler since it's such a common one and benefits from extra work done to make it more readable.

Kernels are built-in interpreter functions. Print is defined in the game scripts.

Here, I built a version with the fix for the Print issue:
http://scicompanion.com/Downloads/SCICompanionAlpha.zip

[Charles]

Here, I built a version with the fix for the Print issue:
http://scicompanion.com/Downloads/SCICompanionAlpha.zip

404 error... is that me, or is the file offline?

[troflip]

Sorry... fixed the link...

[Charles]

Wow, you're awesome! That was an amazingly fast turnaround fixing the Print decompiling.

Now a clean QFG1-EGA decompiles to:

Code: [Select]

(procedure (proc0_27)
(= global4
(Print
"How about a slice of quiche?"
#title
{Giving up, huh?}
#button
{Quit}
1
#button
{Don't Quit}
0
#icon
800
1
4
)
)
)
It didn't work when I tried re-decompiling code I'd made a bunch of changes to, but I don't really have much time right now to debug that, so I'm just gonna forge ahead for now.  Maybe tomorrow I'll be able to try copying my sco files to a fresh QFG1 and see if that works or not.

So, now that code block brings up a couple questions... I found reference to the #'s in the Selector documentation:

http://scicompanion.com/Documentation/Compiler/selectors.html
Selectors are identifiers for properties and methods of classes and instances. Each property and method has a numerical selector index, which points to its name in vocab.997.
(...)
You are also able to retrieve the numerical selector index of a property or method selector.

Which, when I look in vocab.997, I see that #title is 80, #button is 81, and #icon is 82, so that all makes sense.  Can I reference those anywhere in code, including asm blocks, and the compiler will replace it with the index lookup value found in vocab.997?  And then the interpreter will treat that as either a property/method index or an integer as appropriate?  (It's not like there are any strongly typed variables here...)
Is it possible to add a tooltip that shows the index value when hovering over #selectors? Even if they're only used like constants for Print procedures, that's still a very commonly used procedure.

Then there's the "" vs {}. I  read somewhere here that the original Sierra compiler would automatically convert "" strings into Text resources and leave {} string inline, but the last I read, SCICompanion did not duplicate that behaviour.  Does this mean it does now?

I assume Text resources are generated in the order they appear in script.So, does that mean that if I were patching a game (either my own release, or a Sierra game) and I had changed the order of some things in the script, I would need to distribute the new script.xxx and text.xxx files?

[troflip]

Which, when I look in vocab.997, I see that #title is 80, #button is 81, and #icon is 82, so that all makes sense.  Can I reference those anywhere in code, including asm blocks, and the compiler will replace it with the index lookup value found in vocab.997?  And then the interpreter will treat that as either a property/method index or an integer as appropriate?  (It's not like there are any strongly typed variables here...)

A symbol that is preceded by a pound/hash sign (#) does a selector lookup and replaces the selector name with the number. So #title will evaluate to 80, #button to 81, and so on. It should work anywhere a value like a number (or a string, etc..) can be used. Literally all the compiler does is lookup the number of the selector and replace it with that number.

You'll see this used a lot in code that operates on lists. For example, "call this method on every object in this list":

Code: [Select]

; Call dispose on every member of the cast.
(gCast eachElementDo: #dispose)

... you use the # notation to pass the method number to the eachElementDo method.

The fact that these are used in the Print procedure is just kind of random. They could easily have used hard-coded enums too. There's no #at or #button or #title method ever being called. The Print procedure just uses them as identifiers to indicate what the next few parameters are.

Is it possible to add a tooltip that shows the index value when hovering over #selectors? Even if they're only used like constants for Print procedures, that's still a very commonly used procedure.

Yes, should be easy.

Then there's the "" vs {}. I  read somewhere here that the original Sierra compiler would automatically convert "" strings into Text resources and leave {} string inline, but the last I read, SCICompanion did not duplicate that behaviour.  Does this mean it does now?

I don't know if Sierra ever did that - I know you'll find quotes of me saying that, and claiming that in the documentation, because I heard it somewhere . But we've seen no evidence of that in all the documentation that Omer has surfaces, as far as I know.

And yes, SCI Companion does support that now, but you have to turn it on explicitly for each script (since overwriting text resources when you compile a script is "risky" default behavior). Open the SCI Companion help and search for "auto generation". There should be one hit, and it's explained on that page. This isn't in the online help yet, because the official release from back in January does not have that functionality. So you have to use the help from the Help menu.

I assume Text resources are generated in the order they appear in script.So, does that mean that if I were patching a game (either my own release, or a Sierra game) and I had changed the order of some things in the script, I would need to distribute the new script.xxx and text.xxx files?

Yes, you would, if you enable the auto-generation of text resources. Or you could just leave it disabled as it is by default - then the strings will be compiled directly into the script of course, since the decompiler has replaced the resource/num tuple with a string (bloating the script resource, and potentially causing out of memory errors). Then the text resource won't even be used by the recompiled script.

Actually, enabling auto-generation of text resources is risky for a decompiled game, since I can't identify every place they are used. All the decompiler has to go on is two numbers in a row, and it only makes the assumption that they are a text resource tuple if the procedure call is "Print". So it'll miss some if they are used elsewhere, and then those tuples will be invalid once the text resource is auto-generated by compiling the script. So the game would show the wrong strings or crash.

Perhaps I should include an option in the decompiler to not replace the tuples - leave them as is. Maybe try to put a comment next to the Print call or something that contains the text, so it's still easy to follow a script's logic.

[Kawa]

Maybe try to put a comment next to the Print call or something that contains the text, so it's still easy to follow a script's logic.

That's what I do in my decompiles.

[OmerMor]

I think a comment is better than changing the script to use inline strings instead of text resources.

[troflip]

Potential new piece of syntax to get the size of an array:

Code: [Select]

(local
; implicitly sized array (currently supported)
myArray = [1 2 3 4]
)

And now instead of this:

Code: [Select]

(= i 0)
(while (< i 4)
(FormatPrint "hello %d" i)
(++ i)
)

You can do this:

Code: [Select]

(= i 0)
(while (< i &sizeof myArray) ; compiler knows the size of the array
(FormatPrint "hello %d" i)
(++ i)
)

The benefit being, you don't need to hardcode a value for your loop. If you change the number of elements, the number will change.

Similar benefit for explictly-sized arrays too, of course:

Code: [Select]

(method (init &tmp i [someArray 6])
    (for ((= i 0) (< i &sizeof someArray) ((++ i))
        ; ... something
    )
)

Currently, you'd need to make a define so you don't repeat the number twice (and risk it getting out of sync as you make changes):

Code: [Select]

(define SOMEARRAY_SIZE 6)

(method (init &tmp i [someArray SOMEARRAY_SIZE])
    (for ((= i 0) (< i SOMEARRAY_SIZE) ((++ i))
        ; ... something
    )
)

Not sure about the exact syntax yet...

[Kawa]

I like the &sizeof thing. Little things like that.

About the Print inlining, I've been looking through Police Quest 2 a bit and there's a minor snag:

Code: [Select]

(procedure (localproc_0060)
(Print &rest #at -1 15)
)

; ...

((Said 'display,flash/badge')
(if (gEgo has: 7)
(localproc_0060 20 2) ; would be (localproc_0060 "\"I didn't do it! She made me!\"")
else
(localproc_0060 20 3) ; would be (localproc_0060 "You don't have your identification with you.")
)
)It doesn't exactly work when you're using a wrapper like that. Of course, it's perfectly understandable why this is. Other'n that, I dig it.

Fun fact: rm20 has exactly two Print calls -- the wrapper and the narration when you draw a gun on the guard.

[troflip]

It's not possible to know if two numbers passed to a procedure are meant to be resource tuple, so it is (was) hard-coded to only check for Print. I've changed it (in a forthcoming release) so that it can be configured in the Decompiler.ini:

Code: [Select]

[textResourceTuples]
textResourceTuples = [ "Print", "FormatPrint" ]

And also it will by default simply add a comment with the text, instead of replacing with the text.

[Kawa]

I wonder how that'd work with local procedures, if at all.

[troflip]

It matches by procedure name, so as long as you find out the name of the local procedure by decompiling first (the given name just involves it's offset in that script resource), then you could add it to the decompiler.ini and re-decompile.  Decompiler.ini is copied to the src folder (if it doesn't already exist) whenever you open the decompile dialog, and read in every time you do a decompile... so any changes should be picked up and you can basically customize it for a particular game.

I don't think I ever made any documentation for the Decompiler.ini file, because it's just meant as an "escape valve" to allow customization if needed. If you put bad things in there, it might cause SCI Companion to crash, because I'm not very rigorous about validating its contents.

But the info in Decompiler.ini is how Companion knows, for instance, that the signal property should be expressed in hex:

Code: [Select]

(instance Ball1 of Act
(properties
y 99
x 183
view 136
signal $4000
illegalBits $0000
)
)

Or that the first parameter to a handleEvent method should be called pEvent, or to a changeState should be called newState:

Code: [Select]

(method (changeState newState &tmp gloriaX_2)
(switch (= state newState)
...

... that the parameter passed to a respondsTo method call should be a selector literal:

Code: [Select]

((param1 respondsTo: #nsLeft)

... or that code that compares numbers to the type property of a variable called pEvent should turn the numbers into event defines:

Code: [Select]

(switch (pEvent type?)
(evKEYBOARD
...


Instead of hard-coding that "magic", I put it in Decompiler.ini so it could potentially be modified by someone.

[OmerMor]

Tying procedure names to this text matching functionality could be a bit problematic because each function might have that tuple located elsewhere in its signature.
Maybe it would be possible to somehow annotate parameters in a way that would hint the decompiler?
Of course, the annotation should "survive" future decompilations so would probably need to be added to the SCO file, like the procedure names and their signature.

Here's an example of how ReSharper is using custom annotations to hint it about functions that format strings:
http://www.jetbrains.com/help/resharper/2016.1/Code_Analysis__String_Formatting_Methods.html#custom

[troflip]

That would be nice functionality to have, for sure.

Tying procedure names to this text matching functionality could be a bit problematic because each function might have that tuple located elsewhere in its signature.

I don't think I've ever seen a case of that... seems like it's always the first two parameters.