Everybody drop everything and see this (100% SCI decompiled sources!)

[MusicallyInspired]

Sluicebox, a ScummVM developer, has created his own SCI decompiler that now can 100% decompile scripts (including blocks of code that up until now have currently fallen back to assembly) and also added commented annotations within the source to describe what each object is and what it does. It also puts strings from messages as inline comments so you can more easily see what's going on at a glance. I can't wait to play with this later tonight!

https://www.benshoof.org/blog/sci-scripts

Here's OneShortEye's video which just broke the news. SluiceBox has been working very closely with the Sierra speedrunning community to diagnose certain exploit behaviour etc. This is wonderful news!!

NOTE: He doesn't supply the decompiler itself, but he has shared all of the 100% complete decompiled sources for every version of every SCI game that he could get his hands on. He didn't create his decompiler in C++ so it's not compatible with SCI Companion, at least in its current state. But maybe we can convince him to release the source anyway and someone brave enough can adapt it . There are a few other caveats he goes over in the article as well. One namely being that there might be a couple bugs that won't compile but could be fixed by going over the code. There was also something he said about a holdover bug from Sierra's original compiler.

Maybe! If SCI Companion fully supports the game, then it can compile most scripts. You may need to fix a Sierra bug or two, or make some other adjustments to make SCI Companion happy. My goal is to show what's really going on in the scripts, bugs and all.

[Kawa]

I noticed the repo this morning. So this is what that's all about.

(There's no way he wrote his decompiler...)

[Collector]

Nice find.

[MusicallyInspired]

I noticed the repo this morning. So this is what that's all about.

(There's no way he wrote his decompiler...)

Why the skepticism? He's one of the main SCI ScummVM devs.

[Kawa]

My skepticism has been cured. I forgot Phil explained the whole damn process on his blog.

Which is silly cos it's a good read.

[doomlazer]

Awesome that he's achieved 100% decompile. Super useful and quite an accomplishment. I hope to see SCICompanion eventually achieve the same.

I will point out that the video seems to overstate the code annotations a bit. I was hoping to see all generic variable and procedure names replaced, but that is not the case - at least from looking at PQ2.

Many of the common (shared?) variables & procedures have been changed, but the majority of global and local vars still need to be manually reverse engineered. It also fails to recognize local print procedures, so those messages do not get appended in the scripts.

I don't mean to distract from the 100% decompile achievement. There is still plenty of legwork needed to really understand a lot of the code, but it's certainly another valuable resource.  Grats, Sluice.

Edit: One nice feature is that his code doesn't seem to always use the first word in a group, so you see "woman" in the code instead of the more off putting "broad" everywhere.

[Collector]

It would be nice to see the source for his decompiler, not that I could do much with it. Or at least a build of it.

[doomlazer]

I did a little testing with PQ2 v1.002.011. After copying over the scr folder and .ini from github and fixing a few minor issues I was able to compile in SCICompanion. Unfortunately, it seems to introduce the same new bugs as the ones that decompiling in SCICompanion produces.

Since Sluice's github doesn't have an issue tracker, and we get the same problems from decompiling in SC, I'll mention them here instead:

At the start of the game, Sonny's car pulls in and transitions to the interior of the personal car. TAKE KEYS, then EXIT. The scene returns to the parking lot (rm1), but sonny never exits the vehicle - ego is at posn: 0 0. This can be mitigated by changing the following in rm1::doit

Code: [Select]

...
((== global132 local5 1)
;(if (not (gCast contains: ourCar))
(if (gCast contains: ourCar)
(= global132 0)
(self setScript: exitScript)
)
)
...


Now that Sonny can get out of the car correctly, we run into a very common bug with a decompiled version of PQ2:

With the keys from the car Sonny should be able to OPEN DOOR to the police station, but because the autodoor doesn't animate it never reaches cel 2 and Sonny is stuck.

Generally these autodoor animation failures are related to "stopUpd:", removing them will often fix the issue, but in this case I didn't test enough to resolve the problem.


Edit: In case anyone wants to test (or verifiy I'm not introducing the bugs) here are the items I had to correct to get Sluices PQ2 1.002.011 source to compile in SC 3.2.4:

Error: (Main.sc) Undeclared identifier 'sndVOLUME' .  Line: 663, col: 11

Code: [Select]

;(DoSound sndVOLUME 15)
(DoSound 8 15)


Error: (Main.sc) &rest cannot be used if the send target itself contains nested procedure calls or sends. Assign the result of the procedure call or send to a temporary variable and use that instead.  Line: 411, col: 27

Code: [Select]

;comment out the entire procedure because it's unused or fix like so:

;;;(procedure (Notify param1) ; UNUSED
;;;    ((ScriptID param1) notify: &rest)
;;;)
(procedure (Notify param1 &tmp temp0) ; UNUSED
(= temp0 (ScriptID param1))
(temp0 notify: &rest)
)

Error: (rm8.sc) The rest modifier must be followed by a parameter name.  Line: 56, col: 23

Code: [Select]

;(Display &rest dsCOORD 73 local143 dsFONT 7 dsCOLOR 0 dsBACKGROUND 0)
(Display &rest 100 73 local143 dsFONT 7 dsCOLOR 0 dsBACKGROUND 0)

Error: (Introduction.sc) The rest modifier must be followed by a parameter name.  Line: 31, col: 32

Code: [Select]

;(= temp1 (Display &rest dsALIGN alCENTER dsCOORD 10 10 dsWIDTH 300 dsCOLOR 15 dsSAVEPIXELS))
(= temp1 (Display &rest 101 alCENTER dsCOORD 10 10 dsWIDTH 300 dsCOLOR 15 dsSAVEPIXELS))

Error: (Interface.sc) Unknown procedure 'proc999_4' .  Line: 35, col: 13

Code: [Select]

; procedure is unused just block comment it out:

;;;(procedure (MousedOn obj event theMods) ; UNUSED
;;; (cond
;;; ((!= (event type:) 1) 0)
;;; ((and (>= argc 3) theMods (== (& (event modifiers:) theMods) 0)) 0)
;;; ((obj respondsTo: #nsLeft)
;;; (proc999_4
;;; (obj nsLeft:)
;;; (obj nsTop:)
;;; (obj nsRight:)
;;; (obj nsBottom:)
;;; event
;;; )
;;; )
;;; )
;;;)

;and the public export:

(public
Print 0
ShowView 1
GetInput 2
GetNumber 3
Printf 4
;MousedOn 5
StillDown 6
)

Error: (Game.sc) Undeclared identifier 'sndRESUME' .  Line: 129, col: 11

Code: [Select]

;(DoSound sndRESUME)
(DoSound 7)

Error: (User.sc) Unknown procedure 'proc984_0' .  Line: 101, col: 13

Code: [Select]

;script 984 doesn't exist so comment it out

...
(method (said event)
(if global54
;(proc984_0 alterEgo gSortedFeatures gCast gFeatures)
else
...

Error: (Actor.sc) Unknown procedure 'CosMult' .  Line: 522, col: 41

Code: [Select]

;(= x (+ xOrg (* legLen (sign (CosMult (* legDir 45) 100)))))
(= x (+ xOrg (* legLen (sign (TimesCos (* legDir 45) 100)))))

Error: (Actor.sc) Unknown procedure 'SinMult' .  Line: 523, col: 41

Code: [Select]

;(= y (- yOrg (* legLen (sign (SinMult (* legDir 45) 100)))))
(= y (- yOrg (* legLen (sign (TimesSin (* legDir 45) 100)))))

Some of these will need to be fixed in multiple places, but this should allow "Compile All" to complete without error.


Edit2: a few other changes are need to get things running in ScummVM. In script 989 find and change the following:

Code: [Select]

;in script 989 "Sound"
;in init:
   ;(DoSound sndINIT self)
   (DoSound 0 self)

;in play:
   ;(DoSound sndPLAY self)
   (DoSound 1 self)

in dispose:
   ;(DoSound sndDISPOSE handle)
   (DoSound 10 handle)


[MusicallyInspired]

Nice finetuning there!

[Collector]

Perhaps you should fork that repo and add an issue tracker for fixes and tweaks. Has anyone looked at Sluicebox's decompiled systems scripts and compared them to the Sierra's scripts we have?

[doomlazer]

Perhaps you should fork that repo and add an issue tracker for fixes and tweaks. Has anyone looked at Sluicebox's decompiled systems scripts and compared them to the Sierra's scripts we have?

Well, personally I'd like to see the decompiled asm merged into EO's archive. It's a bit more fleshed out with lots of flags and vars identified. Plus I think the naming conventions are more historically accurate, but I really don't know.

I'm mostly curious about how the bugs are getting introduced somewhere in the decompile/recompile process. It's bad enough that Ken left so many Sierra bugs in the games, we don't need any new ones.

[Kawa]

Plus I think the naming conventions are more historically accurate, but I really don't know.

Oh, they are.

No use blaming the bugs on Ken though.

[doomlazer]

I blame Ken for everything. The boss can take it.

[doomlazer]

At the start of the game, Sonny's car pulls in and transitions to the interior of the personal car. TAKE KEYS, then EXIT. The scene returns to the parking lot (rm1), but sonny never exits the vehicle - ego is at posn: 0 0.

I've found that this bug only occurs when running PQ2 through DOSBox directly from SCICompanion. It makes no sense whatsoever. If you switch the profile in SC to ScummVM the bug doesn't appear. If you launch DOSBox independently of SC it doesn't appear.

It doesn't matter if the game has been decompiled/recompiled or if it's a clean copy from gog/steam - launching PQ2 from SCICompanion with DOSBox will cause this issue. I've replaced the dosbox version and config SC uses and it's always the same in both 3.0.1.7 & 3.2.4+.

It still might be a problem specific to my machine as I just have access to one windows computer at the moment, but it still doesn't seem to make much sense.

 

[cosmicr]

It's a shame he hasn't supplied the source code.  He says it's not written in C++, I wonder what language though.