Community
SCI Programming => SCI Community How To's & Tutorials => Topic started by: lwc on September 25, 2023, 03:42:13 PM
-
A similar question on ScummVM's forums (https://forums.scummvm.org/viewtopic.php?t=14739) didn't get a proper reply, but their forums in general aren't too responsive, so maybe there'll be more luck here.
So ScummVM's SCI patches go in https://github.com/scummvm/scummvm/blob/master/engines/sci/engine/script_patches.cpp but maybe since there's no guide there's practically 99.9% (especially in recent years) rely on a single person (the one and only sluicebox, who comes and goes as he pleases and almost never communicates).
Can you a short and easy guide how to get any SCI patch to ScummVM?
-
Well, if you look at that script_patches.cpp file it appears each fix has two components, a signature and a patch:
static const uint16 camelotSignaturePeepingTom[] = {
0x72, SIG_MAGICDWORD, SIG_UINT16(0x077e), // lofsa fawaz <-- start of proper initializion code
0xa1, 0xb9, // sag global[b9h]
SIG_ADDTOOFFSET(+571), // ...
0x39, 0x7a, // pushi 7a <-- initialization code when walking automatically
0x78, // push1
0x7a, // push2
0x38, SIG_UINT16(0x00a9), // pushi 00a9 - script 169
0x78, // push1
0x43, 0x02, 0x04, // callk ScriptID
0x36, // push
0x81, 0x00, // lag global[0]
0x4a, 0x06, // send 06
0x32, SIG_UINT16(0x0520), // jmp [end of fawaz::handleEvent]
SIG_END
};
static const uint16 camelotPatchPeepingTom[] = {
PATCH_ADDTOOFFSET(+576),
0x32, PATCH_UINT16(0xfdbd), // jmp [to fawaz::doit] (properly init peepingTom code)
PATCH_END
};
Without some understanding of both assembly and C++ it would be very difficult to convert.
-
Although ScummVM should use any SCI patch files unless it has been blacklisted the way the NRS SQ4 patch was.
-
So I've written a grand total of one script patch. But to answer your question, if you are looking to make your changes in source and then "convert" to a script patch, then that is not possible. Script patches are very carefully crafted to use the same number of bytes as the unpatched code (or else to jump past some code that it didn't before), and a compiler won't do that. In the case of my one patch, I was able to craft the patch such that it was just a reordering of the original instruction stream (that's a rarity!), and no SCI compiler would generate what I wrote (since it violates the SCI parenthesis structure). Those are only a few of the challenges involved in writing script patches.
-
Although ScummVM should use any SCI patch files unless it has been blacklisted the way the NRS SQ4 patch was.
Yes, the goal is so others will enjoy too (and by now they're unlikely to apply external patches).
So I've written a grand total of one script patch. But to answer your question, if you are looking to make your changes in source and then "convert" to a script patch, then that is not possible.
So if one wants to make a ScummVM patch, one shouldn't even do it in SCI language in the first place? Sounds very hard, since the game's language is SCI and in that case you have to see 1 language and then make a change in another.
-
Yes, the goal is so others will enjoy too (and by now they're unlikely to apply external patches).
No big deal. All that needs to be done in most cases is to simply drop the patch files in the game's directory. SCI was specifically designed to do this. Only the first couple of SCI, like the initial releases of KQ4 and LSL2 didn't recognize resources outside of their own volume files. It allowed Sierra to distribute small, easily applied patches for already released games.
-
Yes, but almost no one in the world will do it (as in look for that patch online to begin with).
But if it's part of ScummVM it'll just be built-in.
-
Which will not work for anyone not using SVM and a large number of Sierra much fans prefer using DOSBox to ScummVM for a number of reasons. SVM does have it place, but often is not the solution that is preferred by many. Besides, most of the patches usually are already included or distributed with the games. In the case of my new installers, the patches are included with the installers.
-
Either way by now most people will only use built-in patches (in distribution or also inside ScummVM).
-
So if one wants to make a ScummVM patch, one shouldn't even do it in SCI language in the first place? Sounds very hard, since the game's language is SCI and in that case you have to see 1 language and then make a change in another.
Not really. SCI games are not distributed as source, but as PMachine binary code. That binary code is what ScummVM (and SSCI) executes, and that binary stuff is what we're changing. The technique is akin to security exploit creation, copy protection cracks and things of that nature that can't be automated. It is best suited for small changes because of the constraints. And yes, some patches are very version dependent, as the comments in that ScummVM file describe.
-
But still ScummVM developers need to take some kind of existing code like "if clicked on chair then stand" and modify it to let's say "if clicked on chair then sit", I'm not sure how they they do it directly in binary.
-
I'm not sure how they they do it directly in binary.
Same way you'd break copy protection or whatever. Disassemble the bytecode into a more readable but still 1:1 form and know as exactly as possible what it's doing.
Let's see an example.
Bytecode:
7A
74 007B
8B 00
35 01
02
36
46 0399 0001 04
PMachine translation:
push2
lofss $007B
lsl 0
ldi 1
add
push
calle 921 1 4
Source text:
(Printf "it's %d" (+ aLocal 1))
Explanation:
- The Printf call takes two arguments, so we push a 2 onto the stack as our argc.
- We load an offset to a string to the stack. This points to "it's %d". The stack now has this on top, then the argc.
- We load the value of the first local variable, aLocal, to the stack.
- We now load a 1 directly into the Accumulator register. The stack is unchanged.
- The add opcode takes the top value off the stack (our local var) and adds it to the Accumulator.
- We now push the Accumulator's value to the stack, making aLocal plus 1, a string pointer, and an argc of 2.
- Finally, we call an external function from script 921, export entry 1, and tell it to look four bytes back on the stack for the arguments.
So if you want to make it subtract, you change that 02 (add) to a 04 (sub).
If you want it to always return 42, replace the entire 8B 00 35 01 02 36 block with 39 2A 00 00 00 00, or pushi 42 followed by what's effectively a no-op (repeated, even number of "binary not" leaving the Accumulator effectively untouched).
It's remarkably easy to change for example a room transition so the game goes from the title screen straight to the starting room instead of a copy protection, too. Imagine for a moment you want to skip the copy protection in Police Quest 2. You look at the decompilation and find that room 701 is the copy protection. It ends with this:
(if gQuit
(Print 701 1) ; "Sorry Bonds, you'll need to do better than that!"
else
(gGame restart:)
)
So passing the copy protection makes the game restart. It'll notice it's restarting and instead of going through the opening titles, it'll do a short speed test and start the game proper. Using the Disassemble feature in SCI Companion we can find the equivalent of the above script:
code_01bc
01bc:3a toss
01bd:81 04 lag gQuit
01bf:30 000c bnt code_01ce
01c2:7a push2
01c3:38 02bd pushi 2bd // 701
01c6:78 push1
01c7:47 ff 00 04 calle ff procedure_0000 4 // Print
01cb:32 0008 jmp code_01d6
code_01ce
01ce:38 00ef pushi ef // $ef restart
01d1:76 push0
01d2:81 01 lag gGame
01d4:4a 04 send 4
code_01d6
01d6:48 ret
So what if we were to take that last part and copy it all the way up to the start of the doit method?
(method (doit) // method_009b
009b:35 00 ldi 0
009d:a1 04 sag gQuit //ensure we won't quit on the next tick
009f:38 00ef pushi ef // $ef restart
00a2:76 push0
00a3:81 01 lag gGame
00a5:4a 04 send 4
00a7:48 ret
So basically (method (doit) (= gQuit false) (gGame restart:)). Suddenly, the copy protection skips itself! For other games, it might be as simple as finding a room change and switching a number around, and if that's not at the same level as "if clicked on chair then sit instead of stand", I don't know what is.
And the exact same thing applies to x86/64 code, or anything else really.
Now, one question is, "how do you translate this to a ScummVM patch?" The better question is "why would you?"
-
Either way by now most people will only use built-in patches
Yeah I'm gonna go ahead and disagree with you on that. I don't get your premise of thought process here. If people aren't going to know to go download patch files for their games how would you changing the ScummVM source code improve those chances when your code wouldn't be put into the main branch? Or are you pursuing having your SVM script patches actually added to the main branch?
And again, half the people who play Sierra games don't use ScummVM.
(in distribution or also inside ScummVM).
In distribution? In distribution of what? The easiest way to distribute is with replacement script files.
-
Well, no wonder mainly 1 ScummVM developer has been adding SCI patches all these years, as it sounds a lot tougher than modifying SCI scripts.
Or are you pursuing having your SVM script patches actually added to the main branch?
Yes. The main branch is full of SCI patches, all stored in 1 big file (linked above).
In distribution of what?
I meant like GOG's.
-
GOG has always included patches with their releases, often even fan made patches like the NRS patches. I am the one that did their initial QfG package and it included all of the known patches, both official and fan made. They have since changed over to SVM from DOSBox for a lot of their releases, but a lot of their customers have been demanding that they put back the original executables in those releases so they can play them in DOSBox.
-
With DOSBox being abandonware (and non of its variants reaching its mainstream status), many will probably join GOG sooner or later in embracing ScummVM.
All it takes is for a new Windows version to not support DOSBox and that's it.
-
With DOSBox being abandonware (and non of its variants reaching its mainstream status), many will probably join GOG sooner or later in embracing ScummVM.
All it takes is for a new Windows version to not support DOSBox and that's it.
All I see here is predictions.
And again: no matter if you use ScummVM to run your SCI games or the original interpreters on DOSBox, plain ol' patch files still work the same. I should know, I just loaded up PQ2 in both options and my copy protection bypass kicked in both times with no complaints.
Edit: I looked into the blacklist thing. I can see three sound issues, one music issue, and one musician issue (and that one can be un-blacklisted with a checkbox). The NRS file patches are either disabled by internal SVM patches (speed throttle stuff that's not needed in SVM and may in fact cause more issues), outright baked in (Skate-O-Rama timing), or otherwise unrelated internal patches with alternate versions specifically made to work with NRS versions.
-
All I see here is predictions.
I think GOG switching to ScummVM means it's already happening and not just predictions.
In any case, a patch that will be released nowadays has no way to reach GOG, etc. The only chance it has for having more than 5 people using it is by being a ScummVM patch.
-
Even if GOG switched to ScummVM, why insist on using ScummVM internal patches? And besides, that's just GOG. If you don't get your sierras from GOG, you decide if you want to play 'em on ScummVM, DOSBox, or an actual contemporary PC. Only reason you might not take the GOG version and run it on DOSBox or an actual old beige box is if they only include the resource files and leave out the interpreter and drivers since ScummVM doesn't need those.
"All it takes is for a new Windows version to not support DOSBox and that's it." do you realize how unlikely that is? The Windows devs wouldn't need to just drop the ball, they'd have to drop three dozen to break compatibility that hard. And DOSBox is not just available for Windows, let's not forgot, any more than ScummVM is.
Edit: the same can be said about ScummVM to be entirely honest. If you can't guarantee a future version of Windows will still be able to run DOSBox, you can't guarantee it'll still run ScummVM either.
Pros of internal ScummVM patches: they can be bound to specific versions of specific games via signatures.
Cons: every time you make one, you have to update ScummVM and all affected users have to download a whole new copy of ScummVM, because they're inherently a part of ScummVM.
Pros of external patch files: they're easy to make (just use SCI Companion and export the affected area), they can cover most any type of resource instead of only scripts, they're nice and small compared to the whole SVM.
Cons: you can't automatically match a given patch to a specific version of a specific game (kinda re like IPS vs BPS in that regard).
...I feel kinda inspired to add signature matching support to SCI11+ now. A special header (if the first byte's not & 0x80 it's not a valid raw patch) to state the target's game ID and the bytes to find, followed by the replacement. 11 2B ; "11+" header, as opposed to 83 00 for a text resource.
4B 51 35 00 00 00 00 00 ; KQ5
01 00 ; one block to replace
09 00 ; nine bytes of signature
11 00 ; seventeen bytes to replace
70 6F 69 73 6F 6E 6F 75 73 ; 'poisonous'
76 65 6E 6F 6D 6F 75 73 20 73 6E 61 6B 65 21 22 20 ; 'venomous snake!" ' with a padding space because we're keeping it simple.
Silly example considering KQ5 isn't SCI11 so my terp can't run it but it serves its purpose. If I didn't want to keep it simple I could add block types so the patch could move data around, like "copy 0x1D2 bytes from 0x79 to 0x78", so the replacement can be only eight bytes long.
-
70 6F 69 73 6F 6E 6F 75 73 ; 'poisonous'
76 65 6E 6F 6D 6F 75 73 20 73 6E 61 6B 65 21 22 20 ; 'venomous snake!" ' with a padding space because we're keeping it simple.
;D
-
What an interesting debate.
And DOSBox is not just available for Windows, let's not forgot, any more than ScummVM is.
I doubt many vintage quest players don't use Windows.
Edit: the same can be said about ScummVM to be entirely honest. If you can't guarantee a future version of Windows will still be able to run DOSBox, you can't guarantee it'll still run ScummVM either.
I disagree with that. Unlike DOSBox, which already stopped releasing new versions almost half a decade ago, at least in the near if not distant future ScummVM developers will prepare a version to fit with new requirements or at least release it after they'll get blocked.
-
Not much of a debate when neither party seems willing to give an inch.
But you know what? I think we've gotten distracted from the real question.
Why would you insist on distributing patches as specifically ScummVM patches?
- They only apply to the specific game and version they're meant for.
- They're only as big as they need to be.
- But not everyone can make them -- that means both putting the patch together and having it merged into the ScummVM repo.
- And they only apply if and only if you use ScummVM.
- And they only work for script resources.
Separate file patches though?
- Anyone with a copy of SCI Companion, SV, and/or a hex editor can make them.
- They work for most any type of SCI game resource -- pictures, views, scripts, text, music...
- They apply both in ScummVM and the original interpreters, on DOSBox and original contemporary MS-DOS machines.
- But you can mismatch them, applying them to the wrong game and/or version.
- And you have to replace the whole file even if it's a single byte changed.
Keep in mind, please, that you don't control what other people want to use to run these games, and can't possibly predict how things may or may not stop running in the future.
... and I know it seems hypocritical to try and get the conversation into focus and then saying this, but that's not even getting into fan games. A bunch of old SCI Studio era fan games are listed in the detection tables, but as far as I know the only SCI1.1 fan game in there is my own. And I can't exactly testify to a good experience trying to develop said game using ScummVM. A single hash mismatch and it utterly refuses to run! So personally, I'll stick to DOSBox, thanks. At least it lets me test my builds.
-
With DOSBox being abandonware (and non of its variants reaching its mainstream status), many will probably join GOG sooner or later in embracing ScummVM.
All it takes is for a new Windows version to not support DOSBox and that's it.
DOSBox is NOT abandonware. That is why the official DOSBox has not added the Munt patch into it. Peter does not want associate it with something that requires the MT-32 ROMs to work, given Roland claims the IP for them. For all of that, SVM at one time went through legal troubles from LA for piracy claims, while DOSBox has never had a challenge against it. To equate DOSBox with abandonware is ignorance at best.
To dismiss DOSBox as no longer being developed is silly. It is under constant development, even if there has been no new major release for the last few years. The last half dozen commits were within the last month or so. All you have to do to see the activity around DOSBox is to go to VOGONS. I would note that the sound of DOSBox is more accurate than SVM and the pixel perfect patch makes the graphics better with the right aspect correction.
The reason that SVM has had so many new releases is that it only supports games that have been added one by one. DOSBox will run almost any DOS game and most applications. This allows DOSBox to run many, many hundreds more games than SVM is able to. Plus there are forks that extend its capabilities, even installing Windows 9x in it.
Also, as Kawa notes, DOSBox is multi-platform and is available for almost everything that SVM is. I don't mean to attack you, but your views of the two have been formed with a lack of information. You may prefer SVM and that is fine, but you don't need to misrepresent the other option to justify your choice.
-
I think we can be more civil than that, as I do feel getting more and more attacked in this thread despite you not meaning to. You can just say you don't agree rather than calling me silly (or other constant name calling that were used in previous messages).
Just because there's a deep and passionate community around DOSBox doesn't mean fluent commits are the same as stable versions.
While they are the same for "insiders", most population doesn't look behind stable versions which is why for example Wikipedia first and foremost mentions the last stable date.
-
No no, look again. Collector called dismissing DOSBox silly, not you.
-
Just because there's a deep and passionate community around DOSBox doesn't mean fluent commits are the same as stable versions.
While they are the same for "insiders", most population doesn't look behind stable versions which is why for example Wikipedia first and foremost mentions the last stable date.
[citation needed]
(speaking of Wikipedia)
Also, it doesn't matter what version or flavour of DOSBox you use (and there are many that are under active development, btw, and always will be), SCI games will work on all of them (old or new).