SCI mysteries

[troflip]

I'll use this thread to talk about mysterious or confusing things I've come across while decompiling scripts.

A number of scripts have objects with blank names. This is true in SCI0 as well as later versions. The template game's MenuBar and Control classes don't actually have those names in the original sierra code they came from (either LSL2 or LSL3). Instead, their names are empty. Those are the only classes with empty names in those games (I think).

In SQ5, again there is a class in script 255 with an empty name, which I believe corresponds to the Control class. It inherits from Obj.
There are also 3 other empty-name classes in SQ5:
    - Script 943 has an empty-name class that has methods like draw/save/restore, and calls kernels like Graph. I don't know what it does.
    - Script 948 has two empty-name classes that appear to be related to debugging (a "feature writer").

These last 3 empty-name classes inherit from 0xffff, which means they inherit from nothing, just like the Obj class.


Does anyone have any guesses as to why Sierra generated scripts with empty class names? For the decompiler, I can just give them generated names, like "Class_255_0". The latter 3 classes mentioned above all appear to be related to debugging, so I can see why they might have their names removed. But the first one is the Control class. I guess it's not actually used outside that file, so maybe it doesn't name a name. Maybe it was a way of marking the class "private" or something.

[lance.ewing]

Regarding the empty names, I think it would have been to save a little (tiny) bit of space. The compiler and language probably supported it and then I guess it was up to the developers as to whether they chose to "blank" or shorten the name. It is already fairly clear to me that some of the class names are shortened from their original, such as PV, which is clearly the PicView mentioned in the Official Book of King's Quest, and Act, which is Actor. Obj itself could be short for Object. Anything that is abbreviated or truncated is likely to be a shorter version of the original source name.  Some other examples:

Rm = Room
Rgn = Region
SL = StatusLine
Rev = Reverse
Fwd = Forward
CT = CycleTo
Blk = Block

Given that the "name" is a property (admittedly a special property), it's default value could probably be overridden, even with a blank value it would seem. The very existence of blank names is almost proof that there was a way to override the name property.

The interpreter itself doesn't really need the names, so I'm assuming that the name property was mainly for debugging purposes.

What also seems clear is that they didn't use this ability to override and shorten the name property all the time. A lot of the classes clearly have their original names, such as Feature, Script, View, Prop, Event, List, Set, Motion, MoveTo.

Regarding inheriting from 0xffff (i.e. nothing), this suggests a super class even more super than Obj that exists within the interpreter kernel itself. For a while I've been wondering if it was called RootObj since this string appears within the SCIV.EXE, but I can't seem to make the interpreter or its debugger display this string. Does anyone know when the interpreter would display this? I show its context within the SCIV.EXE below:

Objects
Fragments
Resources
Free Heap
Proc
  size:%d
  max:%d
  cur:%d
PMach
  size:%d
  max:%d
  cur:%d
Stack
%02x%c
%c %c
Inspect :: $%04x
RootObj
super: %s
%s:%d 
%s:$%x 
Inspect :: %s ($%x)
%s not a Collection.
%s is empty.
%s :: %s
Selector
%s: not selector for %s.
Property

[troflip]

I definitely agree with you about the shortened names (indeed, in the original SCI syntax thread, the feature writer I mentioned outputted full names like "Actor").

As for getting RootObject to be displayed, I haven't been able to. I tried displaying an object in the debugger which had no super class (e.g. the Obj class itself, which inherits from "root"). It's super was displayed as "I5".

Knowning that 0xffff is the marker for "root", I look at the address 0xffff, which, when it wraps around to 0x0000, had this address in it:
0x7573

And looking at that address, and interpreting it as an object, down where the value for the name selector would be, I see:
49 35 00

which is the ASCII string "I5".

So it was just showing whatever garbage happened to be at the address pointed to by the 0xffff address.

So that all makes sense, but it still doesn't help learning what "RootObject" is.

[lance.ewing]

The feature writer might be where I saw PicView actually, rather than the book.

If someone had enough time, they could probably disassemble SCIV.EXE and see where the RootObj string is referenced. It appears among all the other debugger text but the inspector doesn't seem to recognise that name. I wonder if an early version of the interpreter displayed it as the super of Obj and it subsequently broke. It's certainly one of the mysteries.

If we identify all of the classes that inherit from 0xffff, and we then look at what is common between them, we could probably identify what the root obj provides. I'm guessing it would be all the "special" properties, such as species, super, name, -info-.

[troflip]

Interestingly, the long names Actor and PicView are used in most SCI1.1 games.

New mystery:
My decompiler is barfing on invalid offsets in some scripts' export tables. Zero often appears in a script's export table, which I think I mentioned before. However, I've now seen the value 0x3af appear in a few script's export tables (in SQ5, scripts 16 and 104), and it points to garbage (instead of a valid code or an object). It's easy to detect this when the script's code section is shorter than 0x3af (since I can see it's out of bounds), but more difficult when it does point into the middle of valid code.

SCI Viewer shows the export:

Code: [Select]

export104_0 = rm104
export104_1 = $0
export104_2 = $0
export104_3 = $0
export104_4 = $0
export104_5 = $0
export104_6 = $0
export104_7 = $0
export104_8 = $0
export104_9 = $0
export104_10 = $0
export104_11 = $0
export104_12 = $0
export104_13 = $0
export104_14 = quirkTalker
export104_15 = code_03af

3af is a valid script number (943, ClickMenu). I'm not sure if that's important.

[lskovlun]

I think after your latest post I may be able to provide some help here. When we first implemented SCI01 support in FreeSCI, it seemed as though every other export was zero, leading to the (unsubstantiated) hypothesis that interesting things might happen if that "other" export was not zero. Do you recognize this pattern? There definitely seems to be some additional code present in the post-01 SSCI script loader to handle that case; I haven't bothered to look at it, and the scripts seem to work without it (but then again,I'm baffled by some of the workarounds that the ScummVM people claim to be necessary). The pattern from SCI01 seems to be gone in SCI1.1, but it's possible that there are still things to be found in the relocation code (but then you'd wonder why the scripts work in ScummVM at all).

In any case, the 0x3af=943 finding is interesting.

[troflip]

I mean, I don't think the 0x3af is problematic as long as no one calls that export. Like the interpreter shouldn't even ever access that value. But of course it does matter if you're disassembling or decompiling!

It should be easy for me to see if there are other SQ5 scripts with that weird export, and if any code ever calls it (which would be *really* strange).

I haven't seen the "every other export was zero" pattern in any of the games I've looked at yet... just occasional random zeroes in the export table.

[edit:]
Ok, there's only one other script in SQ5 with that 3af export, and it's script 243. I looked at LSL6 and KQ6, and didn't find that 3af export anywhere, so it doesn't look like it's any kind of marker or anything.  And no code in SQ5 calls those scripts' exports.

So I suspect it's just some garbage value that Sierra's compiler wrote out, and it never caused problems, so no one noticed.

[troflip]

Next weirdness, exports appearing multiple times:

Code: [Select]

export209_0 = sStartViewPort
export209_1 = sStopViewPort
export209_2 = viewPort
export209_3 = $0
export209_4 = $0
export209_5 = $0
export209_6 = $0
export209_7 = $0
export209_8 = $0
export209_9 = $0
export209_10 = $0
export209_11 = $0
export209_12 = $0
export209_13 = $0
export209_14 = viewPortTalker
export209_15 = $0
export209_16 = viewPortTalker
export209_17 = $0
export209_18 = viewPortTalker


[lskovlun]

I don't think there's anything preventing mentioning an export several times... and SQ5 was the only title (as far as I know) to be implemented in SCI outside Sierra.

[Collector]

Wasn't RAMA done at Dynamix, too? But of course that was SCI3.

[lskovlun]

I wasn't aware of the Dynamix thing, even though I owned the game... but the demo definitely is SCI2.1...

[troflip]

I don't think there's anything preventing mentioning an export several times... and SQ5 was the only title (as far as I know) to be implemented in SCI outside Sierra.

It's the first that I've encountered it. It does present a small problem for decompilation, since the current SCIStudio language doesn't provide any options to explicitly declare the export # for a procedure or object (but I can use the existing (generated) .sco file to track this).

[Collector]

I wasn't aware of the Dynamix thing, even though I owned the game... but the demo definitely is SCI2.1...

Just checked. I was right that RAMA was done at Dynamix. The full game was interpreter 3.000.000.

[lance.ewing]

Interestingly, the long names Actor and PicView are used in most SCI1.1 games.

Yeah, that is very interesting. Are there any other classes using a longer version in later games? I'm wondering if we can identify a few more of the original class names by looking ahead to newer games. I've noticed some of the short names are 3 or 4 letter acronyms, which makes it a bit harder to guess. But if they use the longer names in later games, it might solve the mystery.

Once you establish original class names for various shortened names, you could get the decompiler to swap them in.

[troflip]

When we first implemented SCI01 support in FreeSCI, it seemed as though every other export was zero, leading to the (unsubstantiated) hypothesis that interesting things might happen if that "other" export was not zero.

Ok, now I'm seeing this (SCI Viewer doesn't show it directly, but you can see it in the raw bytes). I don't think it's that every other export is zero, but rather that each export is 4 bytes instead of 2. If that's correct, it should be an easy fix to make, I just need to know how to detect this reliably.

I found the code in ScummVM that handles this:

Code: [Select]

bool exportsAreWide = (g_sci->_features->detectLofsType() == SCI_VERSION_1_MIDDLE);
So I think I should be able to identify this case.