Community
SCI Programming => SCI Syntax Help => Topic started by: Charles on May 02, 2020, 03:56:12 PM
-
Im working on recreating the source for QFG2, using a mix Eric’s source from the decompilation archive and my own efforts decompiling from scratch.
The problem is I’m seeing what feels like a memory corruption error and I don’t even know where to begin to troubleshoot it.
In a nutshell it looks like variables aren’t being set properly. Take this one instance in script 000, near the end of Trial:init (which isn’t that like the first code the game runs?)
(= possibleScore 500)
(Printf {possibleScore is %d} possibleScore)
It prints the message “possibleScore is 0”
Part of the problem I’m having is that I don’t think there’s anywhere *before* there that I can fix. I haven’t done any extensive troubleshooting, but given what I’m seeing, I don’t know where to begin. Any thoughts?[/code]
-
What does the disassembly say? Script -> Disassemble
-
I wrote the above from memory, and it looks like I got it a bit messed up. It's still more or less the same problem, but it's not in the Trial:init that's causing the problem. Trial:init calls a procedure from another script, that that's the one I'm seeing the problem in.
Trial:init
(method (init &tmp parseStr)
(Bset IN_SHAPEIR)
(= highHunk (if (>= (MemoryInfo TotalHunk) 17408) TRUE else FALSE))
(= systemWindow sfWin)
(= customWindow cWin)
(= ego egoObj)
(= gEgoBase egoBase)
(= version {x.yyy____})
(= waitCursor HAND_CURSOR)
(StatusLine code: (= dftStatusCode statLn))
; Make sure some important modules are loaded in.
;CI: NOTE: This bit here looks suspect... unlike any other SSCI code I've seen.
; ... shouldn't these be in brackets? or assigned to tmp variables or something?
Cycle
StopWalk
DCIcon
Window
TheMenuBar
HandsOffScript
TopWindow
TargActor
(LoadBaseScripts INTRFACE LOADMANY SAVE SIGHT SORTCOPY QG2_DIALOG QG2_EVENTS QG2_TALKER)
;EO: According to SCI16's KERNEL.SH, kernel function 122 is Lock, but it is not in VOCAB.999
;CI: see http://scicompanion.com/Documentation/Kernels/Lock.html?highlight=lock
(kernel_122 RES_TEXT MAIN TRUE)
(kernel_122 RES_TEXT QG2_COMMONTEXT TRUE)
(kernel_122 RES_TEXT MENU TRUE)
(super init:)
;init common handlers
((= keyDownHandler keyH) add:)
((= mouseDownHandler mouseH) add:)
((= directionHandler dirH) add:)
(= useSortedFeatures TRUE)
(= ftrInitializer ftrInitCode)
(= musicChannels (DoSound NumVoices)) ;CI: NOTE: this gets done in the SetGraphicsSoundInit function anyway.
(= doVerbCode HQVerbCode)
(= egoWalk egoW)
(= egoStopWalk egoSW)
(= egoGrooper egoGL)
(= errorLog errLog)
;set up spell casting code segments
(= codeCastSpells (= codeCastDefaultSpells defaultSpellCast))
(= codeCastAllSpells allSpellCast)
(= codeCastNoSpells noSpellCast)
;init common music samples
((= cSound cMusic) number: 10 owner: self init:)
(= cIcon deathIcon)
((= miscSound miscMusic) number: 10 owner: self init:)
((= spareSound spareMusic) number: 10 owner: self init:)
(SetGraphicsSoundInit) ;calls Script 2, procedure 0.
(= parseStr {*})
(StrAt parseStr 0 0)
(User
alterEgo: ego
prompt: parseStr
blocks: 0
y: 160
verbMessager: verbWords
)
(TheMenuBar init:)
(HandsOn 1)
(= possibleScore 100)
(Printf {possible Score1: %d} possibleScore)
(= startingRoom (SetGameInit)) ;calls Script 2, procedure 1.
(= possibleScore 10)
(Printf {possible Score2: %d} possibleScore)
(theGame newRoom: QG2_SPEEDCHECK)
;the first room we go to is room 98: the SpeedCheck Room.
)
Script 2, procedure 1
(procedure (SetGameInit &tmp nextRoom skill)
;turn on debug by default
;CI: NOTE: This is kinda weird. If I don't explicitly set the suckBlueFrog to FALSE right here, none of the following variables get set properly
;, and the game glitches out when the hero gets low on stamina shortly thereafter.
;(= suckBlueFrog FALSE)
(= deathMusic 10)
(= showStyle HWIPE)
(= possibleScore 500)
(Printf {possible Score(SetGameInit): %d} possibleScore)
(= userFont 300)
(= smallFont 999)
(= bigFont 300)
(= timerStamina 20)
(= timerHealth 15)
(= timerMana 5)
(Bset SAVE_ENABLED)
(= currentDay 1)
(SetGameTime 8)
(= sillyClowns clownsOff)
(= arcadeDifficultyLevel arcadeNormal)
(= gameTimeScale timeNormal)
(= nextRoom 0)
(if (and suckBlueFrog TRUE)
(= skill 0)
(while (< skill NUM_ATTRIBS)
(= [egoStats skill] 80)
(++ skill)
)
(= [egoStats EXPER] 1900)
(= [egoStats HEALTH] (GetMaxHealth))
(= [egoStats STAMINA] (GetMaxStamina))
(= [egoStats MANA] (GetMaxMana))
(= [invNum iGold] 100)
(= [invNum iRations] 10)
(= [invNum iWaterskin] 1)
(= currentWater 10)
(StrCpy @userName {Hero})
(Bset fInMainGame)
(= nextRoom (BlueFrogTP))
else
(Bclr fInMainGame)
)
(if (not nextRoom)
(if (GameIsRestarting)
(= nextRoom rOpeningScroll)
else
(= nextRoom rPiracyNotice)
)
)
(return nextRoom)
)
Script Dissassembly for Script 2, Procedure 1.
// EXPORTED procedure #1 (SetGameInit)
(procedure proc_0099
0099:3f 02 link 2 // (var $2)
009b:35 0a ldi a
009d:a0 009b sag deathMusic
00a0:35 00 ldi 0
00a2:a1 11 sag showStyle
00a4:34 01f4 ldi 1f4
00a7:a1 10 sag possibleScore
00a9:7a push2
00aa:74 109f lofss $114c // possible Score(SetGameInit): %d
00ad:89 10 lsg possibleScore
00af:46 00ff 0004 04 calle ff procedure_0004 4 // Printf
00b5:34 012c ldi 12c
00b8:a1 16 sag userFont
00ba:34 03e7 ldi 3e7
00bd:a1 17 sag smallFont
00bf:34 012c ldi 12c
00c2:a1 1a sag bigFont
00c4:35 14 ldi 14
00c6:a0 0081 sag timerStamina
00c9:35 0f ldi f
00cb:a0 0082 sag timerHealth
00ce:35 05 ldi 5
00d0:a0 009c sag timerMana
00d3:78 push1
00d4:78 push1
00d5:47 01 07 02 calle 1 procedure_0007 2 // Bset
00d9:35 01 ldi 1
00db:a1 6e sag currentDay
00dd:78 push1
00de:39 08 pushi 8 // $8 underBits
00e0:40 0809 02 call proc_08ed 2
00e4:35 00 ldi 0
00e6:a0 00b4 sag sillyClowns
00e9:35 02 ldi 2
00eb:a0 0129 sag arcadeDifficultyLevel
00ee:35 01 ldi 1
00f0:a0 00a5 sag gameTimeScale
00f3:35 00 ldi 0
00f5:a5 00 sat temp0
00f7:81 65 lag suckBlueFrog
00f9:31 65 bnt code_0160
00fb:35 01 ldi 1
00fd:31 61 bnt code_0160
00ff:35 00 ldi 0
0101:a5 01 sat temp1
code_0103
0103:8d 01 lst temp1
0105:35 1e ldi 1e
0107:22 lt?
0108:31 0b bnt code_0115
010a:39 50 pushi 50 // $50 title
010c:85 01 lat temp1
010e:b0 023b sagi egoStats
0111:c5 01 +at temp1
0113:33 ee jmp code_0103
code_0115
0115:34 076c ldi 76c
0118:a0 024a sag global586
011b:76 push0
011c:45 04 00 callb procedure_0004 0 // GetMaxHealth
011f:a0 024b sag gHealth_3
0122:76 push0
0123:45 02 00 callb procedure_0002 0 // GetMaxStamina
0126:a0 024c sag gStamina_2
0129:76 push0
012a:45 03 00 callb procedure_0003 0 // GetMaxMana
012d:a0 024d sag gMana
0130:35 64 ldi 64
0132:a0 0161 sag gTheInvNum_2
0135:35 0a ldi a
0137:a0 0153 sag global339
013a:35 01 ldi 1
013c:a0 0175 sag numWaterskins
013f:35 0a ldi a
0141:a0 0083 sag currentWater
0144:7a push2
0145:5a 0000 029f lea 0 29f
014a:36 push
014b:74 0f6d lofss $10bb // Hero
014e:43 47 04 callk StrCpy 4
0151:78 push1
0152:7a push2
0153:47 01 07 02 calle 1 procedure_0007 2 // Bset
0157:76 push0
0158:40 0ae3 00 call proc_0c3f 0
015c:a5 00 sat temp0
015e:33 06 jmp code_0166
code_0160
0160:78 push1
0161:7a push2
0162:47 01 08 02 calle 1 procedure_0008 2 // Bclr
code_0166
0166:85 00 lat temp0
0168:18 not
0169:31 12 bnt code_017d
016b:76 push0
016c:43 2c 00 callk GameIsRestarting 0
016f:31 07 bnt code_0178
0171:34 02fd ldi 2fd
0174:a5 00 sat temp0
0176:33 05 jmp code_017d
code_0178
0178:34 02fc ldi 2fc
017b:a5 00 sat temp0
code_017d
017d:85 00 lat temp0
017f:48 ret
)
And the outputs I see are:
possible Score1: 100
possible Score(SetGameInit): 100
possible Score2: 10
-
Looks fine... I would suggest running it in ScummVM if you haven't already, it might give some good debug messages. My only thought is there were some differences in the byte code in various different versions of the interpreter, maybe that's getting mixed up, and the disassembly as shown (and compiled by) by SCI Companion is not actually correct for that interpreter version.
-
Thanks for looking at it.
I'll give ScummVM a try. How do I do that though... I've tried editing my scummvm.ini to add it as a game, but it always gives me an error: "Error running game: Game data not found"
It's a v2.1.0 build from last October.
-
Okay, I got it up and running in ScummVM.
And it works perfectly. The score gets set properly, and the game doesn't crash when entering the alleyway. It does crash entering the alleyway when run in the original interpreter.
So, I think you're on to something about byte-code differences. I'm assuming the scummvm interpreter is more flexible about accepting different SCI versions, or something.
EDIT:
I found a difference when comparing the disassembly from the original script.002 and my newly compiled script.002. Is this significant? (This is just a small snippet, from the 1st procedure... The variable names are missing, which I assume is because I didn't generate any sco files for anything.)
Original SCRIPT.002
// EXPORTED procedure #0 ()
(procedure proc_0048
0048:35 00 ldi 0
004a:a1 65 sag
004c:78 push1
004d:39 03 pushi 3 // $3 y
004f:43 2d 02 callk DoSound 2
0052:a1 80 sag
0054:78 push1
0055:7a push2
0056:43 6c 02 callk Graph 2
0059:a1 7f sag
005b:36 push
005c:35 08 ldi 8
005e:22 lt?
005f:30 0014 bnt code_0076
0062:34 032b ldi 32b
0065:a1 99 sag
0067:35 0f ldi f
0069:a1 9a sag
006b:35 00 ldi 0
006d:a1 95 sag
006f:35 04 ldi 4
0071:a1 96 sag
0073:32 0011 jmp code_0087
new compiled SCRIPT.002
// EXPORTED procedure #0 (SetGraphicsSoundInit)
(procedure proc_0048
0048:35 00 ldi 0
004a:a1 65 sag suckBlueFrog
004c:78 push1
004d:39 03 pushi 3 // $3 y
004f:43 2d 02 callk DoSound 2
0052:a0 0080 sag musicChannels
0055:78 push1
0056:7a push2
0057:43 6c 02 callk Graph 2
005a:a1 7f sag colorCount
005c:36 push
005d:35 08 ldi 8
005f:22 lt?
0060:31 17 bnt code_0079
0062:34 032b ldi 32b
0065:a0 0099 sag hpStatusView
0068:35 0f ldi f
006a:a0 009a sag hpFontColor
006d:35 00 ldi 0
006f:a0 0095 sag statColor
0072:35 04 ldi 4
0074:a0 0096 sag statColorNew
0077:33 15 jmp code_008e
So, they look nearly the same... but I think the sag commands on the original one are all 1 byte, whereas the sag commands on the new one are mostly 2 bytes. So, now what can I do with this information?
-
If I remember correctly, most of these opcodes come in 8 and 16 bit variants, with only their lowest bit different. For example, sag.b is opcode A1 and sag.w is A0. For whatever reason the "sag musicChannels" line went from sag.b 80 to the theoretically identical sag.w 0080. I think the bnt in the third block is more interesting, but it too just jumps to after that block in both cases.
-
Does the QFG2 interpreter include the debugger in it? I know it started being removed around that time. If it does, then you could use that (-d on the command line I think, or.... ctrl-shift-D while playing?). If you can set a breakpoint on that function, it will show you each instruction its processing as you step through it.
-
The interpreter that shipped with QFG2 does not include the debugger, however Eric found a compatible interpreter in one of the Xmas cards or demos or something, which he bundled with his decompilation archive QFG2 source.
It’s not an exact drop-in replacement, but it crashes in the same spots as the stock interpreter. The only difference I’d noticed was concerning the cursor. The stock interpreter will let the game use its custom sword and lion head(?) cursors all the time, whereas the other interpreter will often show the built-in arrow cursor and disk drive cursor instead. It drove me batty trying to figure out what I was doing wrong until I realized the interpreter itself was the problem.
Anyway, I’ll try stepping through the code in the debug interpreter tomorrow. Might even be able to compare that, step-by-step with ScummVM’s interpreter too.
-
Aha!
So with this compiled code:
// EXPORTED procedure #1 (SetGameInit)
(procedure proc_0099
0099:3f 02 link 2 // (var $2)
009b:35 0a ldi a
009d:a0 009b sag deathMusic
00a0:35 00 ldi 0
00a2:a1 11 sag showStyle
00a4:34 01f4 ldi 1f4
00a7:a1 10 sag possibleScore
00a9:34 012c ldi 12c
00ac:a1 16 sag userFont
...
QFG2 is reading it fine until except until the instruction at 00a7
Instead of reading it as sag $10 it reads it as sati $65.
Then it carries on at instruction 00a9 as if nothing was wrong.
I haven't checked the actual binary file to see what the code really is, but clicking Script-Dissemble gives me the snippet I pasted above.
-
QFG2 is reading it fine until except until the instruction at 00a7
Instead of reading it as sag $10 it reads it as sati $65.
So, sati is $5a, and both $5a and $65 look suspiciously like ascii characters, so your initial idea about memory corruption seems to be correct. The bytes translate to ascii as 'Ze'.... it doesn't appear in the scripts, so perhaps it's the start of your character's name or something?
EDIT: Wait, no it's not... because the opcodes in ScummVM are shifted one place to the right (operand size in the low bit). Sati would then be either $b4 or $b5. I'll need to think more about this. Memory corruption still seems plausible.
-
I think sati is actually $B8 (wide) or $B9 (byte). And given the command it overwrote was the a single-byte parameter and it didn't overshoot the next command, I assume this sati was $B9.
I'm using this (http://sierrahelp.com/SCI/Wiki/index.php?title=SCI_Specifications:_Chapter_5_-_The_SCI_Virtual_Machine) as a guide.
I confirmed the actual bytes in the file, and (as expected) they matched what the disassembly report showed. The game is changing them after loading them in memory. This helps explain to me why this error would go away if I commented out a line earlier on... maybe it's overwriting whatever's at address 00a7.
I'm a fair bit out of my depth here, but I'm thinking maybe I try commenting out that one line, and keep an eye on that address, see if it still changes values.
-
Which line did your comment out? That could be the clue we need.
-
It's a line in another procedure in the same file.
(procedure (SetGraphicsSoundInit)
;;;NOTE: If this is left in, possibleScore will be changed in the second procedure.
;If it is commented out, then possibleScore will NOT be set properly
(= suckBlueFrog FALSE)
;;;
;;;
(= musicChannels (DoSound NumVoices))
(if (< (= colorCount (Graph GDetect)) 8)
(= hpStatusView 811)
(= hpFontColor vWHITE)
(= statColor vBLACK)
(= statColorNew vRED)
else
(= hpStatusView 810)
(= hpFontColor vLCYAN)
(= statColor vLBLUE)
(= statColorNew vLRED)
)
(DoSound MasterVol 15)
(Joystick JoyRepeat 0)
)
(procedure (SetGameInit &tmp nextRoom skill)
(= deathMusic 10)
(= showStyle HWIPE)
(= possibleScore 500)
(= userFont 300)
(= smallFont 999)
(= bigFont 300)
(= timerStamina 20)
(= timerHealth 15)
(= timerMana 5)
(Bset SAVE_ENABLED)
(= currentDay 1)
(SetGameTime 8)
(= sillyClowns clownsOff)
(= arcadeDifficultyLevel arcadeNormal)
(= gameTimeScale timeNormal)
(= nextRoom 0)
(if (and suckBlueFrog TRUE)
(= skill 0)
(while (< skill NUM_ATTRIBS)
(= [egoStats skill] 80)
(++ skill)
)
(= [egoStats EXPER] 1900)
(= [egoStats HEALTH] (GetMaxHealth))
(= [egoStats STAMINA] (GetMaxStamina))
(= [egoStats MANA] (GetMaxMana))
(= [invNum iGold] 100)
(= [invNum iRations] 10)
(= [invNum iWaterskin] 1)
(= currentWater 10)
(StrCpy @userName {Hero})
(Bset fInMainGame)
(= nextRoom (BlueFrogTP))
else
(Bclr fInMainGame)
)
(if (not nextRoom)
(if (GameIsRestarting)
(= nextRoom rOpeningScroll)
else
(= nextRoom rPiracyNotice)
)
)
(return nextRoom)
)
It's the first line in the first procedure, actually. If I leave it in then sag $10 instruction runs as expected (although there are other oddities in other areas of the game). If I take it out, then sag $10 gets changed to sati $65 (and again, probably other errors elsewhere.
-
Does the SCI interpreter debugger have a "break on write" type of breakpoint? I forget. That would narrow it down pretty quickly.
(also doesn't ScummVM keep code and heap separate, and warn if someone tries to write to script memory?)
-
Oh, that's brilliant. The code/heap seperation would also explain why ScummVm appears to be unaffected, while SSCI is having trouble.
The only warning that ScummVM gives me on my compiled code vs stock QFG2 is:
WARNING: detectLofsType(): failed, taking an educated guess!
Which happens before Trial:init, based on where I placed my SetDebug breaks. Also before Trial:play... which I thought was supposed to be the first thing called?
-
The only warning that ScummVM gives me on my compiled code vs stock QFG2 is:
WARNING: detectLofsType(): failed, taking an educated guess!
This has to do with whether the argument of lofsa/lofss is relative or absolute. It is indeed determined before a single line of script code is run.
-
Hmm... So that's a ScummVM function... I see here (https://github.com/scummvm/scummvm/blob/ee139e16840fb59741dd1120ba715f0e5a4c0daf/engines/sci/engine/features.cpp) that it depends highly on the getSciVersion function.
So ScummVM doesn't detect it as SCI_VERSION_01 (which is what it should be), but has to guess... looks likeit picks either SCI_VERSION_0_EARLY or SCI_VERSION_1_MIDDLE. I'm guessing SciCompanion's compiling it as SCI_VERSION_0_EARLY... but ¯\_(ツ)_/¯ -- still, this kinda circles back to it being a bytecode discrepancy.
Can I force SciCompanion to compile scripts under a different version?
-
You can with Game -> Version Detection. It'll let you reload the game with overridden behavior, including "LOFSA Absolute". But you have to do it each time you load the game - it was only meant to debug version detection issues, and I never got around to persisting the version settings in a file somewhere.
But if the problem was lofsa absolute/relative, the game would crash almost instantly, so i kind of doubt that's the issue...
-
(also doesn't ScummVM keep code and heap separate, and warn if someone tries to write to script memory?)
Also, this is more complicated than that. SSCI made the innovation of separating stuff that needs to be in the heap and stuff that's stored in the hunk. At first, this was done at run time, later (in SCI1.1) at compile time. That is why the SCI01 debugger has those 'es:' and 'si:' displays (ES refers to the x86 segment register and points to somewhere in the hunk, SI likewise is an x86 register). SN stands for 'script number'. ES and SN should change at the same time, and if they go out of whack unexpectedly, that's another clue.
(http://sciprogramming.com/community/index.php?action=dlattach;topic=1761.0;attach=1216)
ScummVM actually stores the script in one piece (in the case of SCI1.1, the script and heap resources are concatenated and stored in one segment). Local variables get their own segment (one per script file), and there is one segment for clones (dynamically created objects), lists and list nodes.
I agree with the part about relative/absolute addressing being an unlikely cause.
-
I'm not necessarily suggesting the SSCI problems I'm having with QFG2 are because of lofsa, but I am thinking more generally that SciCompanion is having a problem compiling SCI01-compatible scripts [see edit 1].
A problem that is just subtle enough it's not outright crashing in SSCI, but is causing ScummVM to detect the SCI version as > SCI_VERSION_01 and < SCI_VERSION_1_1. So one of: SCI_VERSION_1_EGA_ONLY, SCI_VERSION_1_EARLY, SCI_VERSION_1_MIDDLE, or SCI_VERSION_1_LATE.
I don't know what the differences are in compiling for any of those, but that's the only conclusion I'm able to draw.
I don't think the problem is with script.002-- that's just the first indication I had there was a problem. If I run my compiled QFG2 in ScummVM with a stock SCRIPT.002, it still trips up on auto-detecting.
I'd try stepping through ScummVM to see what exactly is causing that detectLofsType warning, but I haven't gotten scummvm to compile on my machine yet.
EDIT 1: Or whatever version of SCI is used by QFG2... I always thought it was called SCI01, but I'm seeing different naming conventions in the ScummVM code.
-
I don't think the problem is with script.002-- that's just the first indication I had there was a problem. If I run my compiled QFG2 in ScummVM with a stock SCRIPT.002, it still trips up on auto-detecting.
The lofsa/lofss autodetection uses script 994 (aka Game.sc) and the Game class in particular. If the methods of that class do not contain even one lofsa or lofss instruction, it'll throw the warning you're seeing. I've no idea how that could happen. A disassembly of that script might help.
EDIT 1: Or whatever version of SCI is used by QFG2... I always thought it was called SCI01, but I'm seeing different naming conventions in the ScummVM code.
Yes, ScummVM needs to make a lot of fine-grained distinctions in versioning but sticks with a relatively small number of version constants. You can try the version command in a debug console to see just how bad it is.
-
EDIT: I posted the two full dissessemblies, but they were too big.
I see a bunch of lofsa commands in each. I'm assuming the line "01d6:72 0b3b lofsa $0d14 // cast" means QFG2 uses relative addresses for lofsa commands -- starting at address $1d6, add 3 bytes for the lofsa command (plus arguments), then jump $b3b bytes, ending up at address $d14.
Other than the order of the classes and instances being different between the two versions, I can't see anything odd.
EDIT 2: Added a recompiled SCRIPT.994 from QFG1EGA. ScummVM does not flag any detectLofsType() warnings in that recompiled game, and again I can't see any significant differences between it and QFG2's.
-
OK, so I've looked at the attached files. There was a transcription error in your original posting; in sag 'with an argument', the argument would get lost. This sequence of files shows that in the originals, the sag/lag parameters are not shown. In the files compiled by Companion, the parameters do show. But this is disassembling, and a very different thing from actual compiling. Also, the Companion files shuffle objects around in a different order than the original compiler. I don't think it's important.
The lofsa/lofss arguments are relative in both compilers (but because of the reshuffling mentioned about the actual numbers are different).
So we don't really learn much from this.