How to easily convert SCI patches to ScummVM?

[lwc]

A similar question on ScummVM's forums didn't get a proper reply, but their forums in general aren't too responsive, so maybe there'll be more luck here.

So ScummVM's SCI patches go in https://github.com/scummvm/scummvm/blob/master/engines/sci/engine/script_patches.cpp but maybe since there's no guide there's practically 99.9% (especially in recent years) rely on a single person (the one and only sluicebox, who comes and goes as he pleases and almost never communicates).

Can you a short and easy guide how to get any SCI patch to ScummVM?

[doomlazer]

Well, if you look at that script_patches.cpp file it appears each fix has two components, a signature and a patch:

Code: [Select]

static const uint16 camelotSignaturePeepingTom[] = {
0x72, SIG_MAGICDWORD, SIG_UINT16(0x077e), // lofsa fawaz <-- start of proper initializion code
0xa1, 0xb9,                      // sag global[b9h]
SIG_ADDTOOFFSET(+571),           // ...
0x39, 0x7a,                      // pushi 7a <-- initialization code when walking automatically
0x78,                            // push1
0x7a,                            // push2
0x38, SIG_UINT16(0x00a9),        // pushi 00a9 - script 169
0x78,                            // push1
0x43, 0x02, 0x04,                // callk ScriptID
0x36,                            // push
0x81, 0x00,                      // lag global[0]
0x4a, 0x06,                      // send 06
0x32, SIG_UINT16(0x0520),        // jmp [end of fawaz::handleEvent]
SIG_END
};

static const uint16 camelotPatchPeepingTom[] = {
PATCH_ADDTOOFFSET(+576),
0x32, PATCH_UINT16(0xfdbd),      // jmp [to fawaz::doit] (properly init peepingTom code)
PATCH_END
};

Without some understanding of both assembly and C++ it would be very difficult to convert.

[Collector]

Although ScummVM should use any SCI patch files unless it has been blacklisted the way the NRS SQ4 patch was.

[lskovlun]

So I've written a grand total of one script patch. But to answer your question, if you are looking to make your changes in source and then "convert" to a script patch, then that is not possible. Script patches are very carefully crafted to use the same number of bytes as the unpatched code (or else to jump past some code that it didn't before), and a compiler won't do that. In the case of my one patch, I was able to craft the patch such that it was just a reordering of the original instruction stream (that's a rarity!), and no SCI compiler would generate what I wrote (since it violates the SCI parenthesis structure). Those are only a few of the challenges involved in writing script patches.

[lwc]

Although ScummVM should use any SCI patch files unless it has been blacklisted the way the NRS SQ4 patch was.

Yes, the goal is so others will enjoy too (and by now they're unlikely to apply external patches).

So I've written a grand total of one script patch. But to answer your question, if you are looking to make your changes in source and then "convert" to a script patch, then that is not possible.

So if one wants to make a ScummVM patch, one shouldn't even do it in SCI language in the first place? Sounds very hard, since the game's language is SCI and in that case you have to see 1 language and then make a change in another.

[Collector]

Yes, the goal is so others will enjoy too (and by now they're unlikely to apply external patches).

No big deal. All that needs to be done in most cases is to simply drop the patch files in the game's directory. SCI was specifically designed to do this. Only the first couple of SCI, like the initial releases of KQ4 and LSL2 didn't recognize resources outside of their own volume files. It allowed Sierra to distribute small, easily applied patches for already released games.

[lwc]

Yes, but almost no one in the world will do it (as in look for that patch online to begin with).
But if it's part of ScummVM it'll just be built-in.

[Collector]

Which will not work for anyone not using SVM and a large number of Sierra much fans prefer using DOSBox to ScummVM for a number of reasons. SVM does have it place, but often is not the solution that is preferred by many. Besides, most of the patches usually are already included or distributed with the games. In the case of my new installers, the patches are included with the installers.

[lwc]

Either way by now most people will only use built-in patches (in distribution or also inside ScummVM).

[lskovlun]

So if one wants to make a ScummVM patch, one shouldn't even do it in SCI language in the first place? Sounds very hard, since the game's language is SCI and in that case you have to see 1 language and then make a change in another.

Not really. SCI games are not distributed as source, but as PMachine binary code. That binary code is what ScummVM (and SSCI) executes, and that binary stuff is what we're changing. The technique is akin to security exploit creation, copy protection cracks and things of that nature that can't be automated. It is best suited for small changes because of the constraints. And yes, some patches are very version dependent, as the comments in that ScummVM file describe.

[lwc]

But still ScummVM developers need to take some kind of existing code like "if clicked on chair then stand" and modify it to let's say "if clicked on chair then sit", I'm not sure how they they do it directly in binary.

[Kawa]

I'm not sure how they they do it directly in binary.

Same way you'd break copy protection or whatever. Disassemble the bytecode into a more readable but still 1:1 form and know as exactly as possible what it's doing.

Let's see an example.

Bytecode:

Code: [Select]

7A
74 007B
8B 00
35 01
02
36
46 0399 0001 04
PMachine translation:

Code: [Select]

push2
lofss $007B
lsl 0
ldi 1
add
push
calle 921 1 4
Source text:

Code: [Select]

(Printf "it's %d" (+ aLocal 1))
Explanation:

• The Printf call takes two arguments, so we push a 2 onto the stack as our argc.
• We load an offset to a string to the stack. This points to "it's %d". The stack now has this on top, then the argc.
• We load the value of the first local variable, aLocal, to the stack.
• We now load a 1 directly into the Accumulator register. The stack is unchanged.
• The add opcode takes the top value off the stack (our local var) and adds it to the Accumulator.
• We now push the Accumulator's value to the stack, making aLocal plus 1, a string pointer, and an argc of 2.
• Finally, we call an external function from script 921, export entry 1, and tell it to look four bytes back on the stack for the arguments.

So if you want to make it subtract, you change that 02 (add) to a 04 (sub).

If you want it to always return 42, replace the entire 8B 00 35 01 02 36 block with 39 2A 00 00 00 00, or pushi 42 followed by what's effectively a no-op (repeated, even number of "binary not" leaving the Accumulator effectively untouched).

It's remarkably easy to change for example a room transition so the game goes from the title screen straight to the starting room instead of a copy protection, too. Imagine for a moment you want to skip the copy protection in Police Quest 2. You look at the decompilation and find that room 701 is the copy protection. It ends with this:

Code: [Select]

(if gQuit
(Print 701 1) ; "Sorry Bonds, you'll need to do better than that!"
else
(gGame restart:)
)

So passing the copy protection makes the game restart. It'll notice it's restarting and instead of going through the opening titles, it'll do a short speed test and start the game proper. Using the Disassemble feature in SCI Companion we can find the equivalent of the above script:

Code: [Select]

code_01bc
  01bc:3a                toss
  01bd:81 04              lag gQuit
  01bf:30 000c            bnt code_01ce
  01c2:7a               push2
  01c3:38 02bd          pushi 2bd // 701
  01c6:78               push1
  01c7:47 ff 00 04      calle ff procedure_0000 4 // Print

  01cb:32 0008            jmp code_01d6

code_01ce
  01ce:38 00ef          pushi ef // $ef restart
  01d1:76               push0
  01d2:81 01              lag gGame
  01d4:4a 04             send 4


code_01d6
  01d6:48                 ret
So what if we were to take that last part and copy it all the way up to the start of the doit method?

Code: [Select]

(method (doit) // method_009b
  009b:35 00              ldi 0
  009d:a1 04              sag gQuit //ensure we won't quit on the next tick
  009f:38 00ef          pushi ef // $ef restart
  00a2:76               push0
  00a3:81 01              lag gGame
  00a5:4a 04             send 4

  00a7:48                 ret

So basically (method (doit) (= gQuit false) (gGame restart:)). Suddenly, the copy protection skips itself! For other games, it might be as simple as finding a room change and switching a number around, and if that's not at the same level as "if clicked on chair then sit instead of stand", I don't know what is.

And the exact same thing applies to x86/64 code, or anything else really.

Now, one question is, "how do you translate this to a ScummVM patch?" The better question is "why would you?"

[MusicallyInspired]

Either way by now most people will only use built-in patches

Yeah I'm gonna go ahead and disagree with you on that. I don't get your premise of thought process here. If people aren't going to know to go download patch files for their games how would you changing the ScummVM source code improve those chances when your code wouldn't be put into the main branch? Or are you pursuing having your SVM script patches actually added to the main branch?

And again, half the people who play Sierra games don't use ScummVM.

(in distribution or also inside ScummVM).

In distribution? In distribution of what? The easiest way to distribute is with replacement script files.

[lwc]

Well, no wonder mainly 1 ScummVM developer has been adding SCI patches all these years, as it sounds a lot tougher than modifying SCI scripts.

Or are you pursuing having your SVM script patches actually added to the main branch?

Yes. The main branch is full of SCI patches, all stored in 1 big file (linked above).

In distribution of what?

I meant like GOG's.

[Collector]

GOG has always included patches with their releases, often even fan made patches like the NRS patches. I am the one that did their initial QfG package and it included all of the known patches, both official and fan made. They have since changed over to SVM from DOSBox for a lot of their releases, but a lot of their customers have been demanding that they put back the original executables in those releases so they can play them in DOSBox.