Author Topic: Recreating complete QFG1 EGA source code  (Read 2279 times)

0 Members and 1 Guest are viewing this topic.

Offline OmerMor

Re: Recreating complete QFG1 EGA source code
« Reply #15 on: June 14, 2016, 03:48:32 AM »
Why not GitHub?

Offline Charles

Re: Recreating complete QFG1 EGA source code
« Reply #16 on: June 14, 2016, 09:09:28 AM »
Let me know if you're interested in trying out a build of Companion that includes a bunch of decompiler fixes. I've been slowly plugging away at decompiler bugs - mainly the insidious ones that silently produce code that is incorrect. So that the "dream" of being able to "recompile all" and expect it to work perfectly is closer...

It's not quite there yet, but I've been able to decompile Laura Bow 1, and recompile all scripts (correcting a few errors that the decompiler can't possibly make sense of, like missing scripts), and it works well enough to to get through all the intro stuff, and walk around a few rooms and talk to people. I just tried QFG1, and there are still about 4 compile errors I get that I should be able to get rid of with a little more decompiler/compiler work.
Yeah, I'd be interested in that.  When you compile LB1, are you able to watch the Intro?  When I tried removing the Copy Protection (and replacing it with QFG1-style "Piracy hurts" dialog) I suddenly wasn't able to view the Intro no matter if I clicked yes or no.

Speaking of compiling QFG1, I hadn't done that before... I was just happily replacing global variable names and public procedures. When I tried to do that yesterday, I wasn't able to compile at all, and it felt like the biggest reason was because the new names had become decync'd with the .sco file.  Actually, speaking of your Manage Decompilation window says:
Quote
Decompiling a script will also generate a .sco file. The .sco file tracks procedure and variable names which are not present in the compiled script. By default they are given names such as "local4"
You may edit the names to make them more meaningful, and they will be picked up the next time you decompile the script.
How do you edit the names?  Or does that mean I just edit the .sc file and if I click decompile again, it will read in those names and use them?  That seems counter-intuitive to "decompile" when I've already got source code... Any way to edit the sco file directly? i.e. is there any documentation on the format?
I'd written q quick'n'dirty program to the more tedious editing... things like renaming script files and updating all use references to it... or doing global search/replace (with warnings if the new string already exists), or adding in include keys.sh and include game.sh if they're not already there.

That's a cool project!
Would you be sharing your results when you're done? We could all learn from your work.
Yeah, I don't really have a problem sharing when I'm done (or as I go along). I'm doing it all in a personal git server now (GitBlitGO, it's super easy to get up and running), so I can probably open up access to that to a few more people.  I think it might be better not putting it on something like GitHub though... it is technically copyrighted material.  I mean, sharing among a small community of people who I'm pretty sure already legally own copies of the games anyway is one thing...

Y'know, speaking of SCICompanion updates, there's a couple fixes (some relatively simple, some maybe not so) for issues I've come across that would greatly improve my workflow... the three big ones:
1) When doing a Compile All, showing the file that is causing an error, not just the line/col nums,
2) being able to middle-click (or wheel-click) on a tab to close it... or at the very least including a Close All option under the Window menu, and
3) having an option to compile as external patch files, rather than automatically rebuilding into resource.000... this one really hurts my source control... I'll compile a script to check I've not made any errors, and it will modify my resource.000 (which is about 2-3 MB in these early SCI0 games, but that whole 2MB gets added to my source control... In theory, it should only add the difference, but I've found difference checking to be spotty with binary files.

Offline troflip

Re: Recreating complete QFG1 EGA source code
« Reply #17 on: June 14, 2016, 12:11:17 PM »
Yeah, I'd be interested in that.  When you compile LB1, are you able to watch the Intro?  When I tried removing the Copy Protection (and replacing it with QFG1-style "Piracy hurts" dialog) I suddenly wasn't able to view the Intro no matter if I clicked yes or no.

Yes, I'm able to now. That was caused by a decompilation bug in proc255_0 (Print).

How do you edit the names?  Or does that mean I just edit the .sc file and if I click decompile again, it will read in those names and use them?  That seems counter-intuitive to "decompile" when I've already got source code... Any way to edit the sco file directly? i.e. is there any documentation on the format?

You edit them from within the decompiler dialog in the box that says "SCO". It doesn't pick anything up from the script files - that would be counter-intuitive. You need to do everything from the decompile dialog. Again, this is in the documentation:
http://scicompanion.com/Documentation/decompiler.html
(I do suggest you read the docs - some of the questions you've asked here before are answered in there)

Brian documented his .sco format. Should be on the wiki somewhere. At any rate, it doesn't really matter, .sco files are generated when you build too. So if you build things in the right order, there shouldn't be any issues. For example, change a public procedure name in in foobar.sc, and then foobar.sco will get that name. Then other scripts that depend on foobar will now compile correctly (assuming they're using the new name). I do have some script dependency tracking stuff in SCI Companion, but I forget if it's linked to "compile all". Not to mention, it won't work if the scripts don't parse because of syntax errors.


2) being able to middle-click (or wheel-click) on a tab to close it... or at the very least including a Close All option under the Window menu, and

Does clicking on the red X not work or something? Or you just want a bigger hit target? (Hmm, I just noticed VS does this... never used that functionality before).

3) having an option to compile as external patch files, rather than automatically rebuilding into resource.000... this one really hurts my source control...

I'd really like this too :P. I wish I'd broken from Brian's compile model earlier and had a chance to do this correctly. It seems like a big change, but I'll look into it, since I'm not the only one frustrated by it. Most of the groundwork is there to make this happen, so maybe it's not as bad as I think to make this change.
Check out my website: http://icefallgames.com
Groundhog Day Competition

Offline Collector

Re: Recreating complete QFG1 EGA source code
« Reply #18 on: June 14, 2016, 12:17:34 PM »
Shouldn't it be just the sc/sco files and not the compiled resource package for your source control?
KQII Remake Pic

Offline Collector

Re: Recreating complete QFG1 EGA source code
« Reply #19 on: June 14, 2016, 12:21:12 PM »
Why not GitHub?

Last I looked GitHub did not have a free account and my Bitbucket account allows for unlimited repos.
KQII Remake Pic

Offline troflip

Re: Recreating complete QFG1 EGA source code
« Reply #20 on: June 14, 2016, 12:29:41 PM »
I used github for SCI Companion and I didn't pay anything (I think it's free for open source stuff only?). And AFAIK there are no limits on number of repos.
Check out my website: http://icefallgames.com
Groundhog Day Competition

Offline Charles

Re: Recreating complete QFG1 EGA source code
« Reply #21 on: June 14, 2016, 01:56:58 PM »
Again, this is in the documentation:
http://scicompanion.com/Documentation/decompiler.html
(I do suggest you read the docs - some of the questions you've asked here before are answered in there)
I'll give that another read. I'd thought I'd read through most of the docs before, but that was before I'd started diving in and I could clearly use a refresher.

You edit them from within the decompiler dialog in the box that says "SCO".
I swear I'd tried that, but I'm trying it now, and I must not have been patient enough with it before.  I tried selecting the line items, I tried double-clicking on it, but I never tried selecting it then single-clicking it (or pressing F2). And now that I do, I see what appears to be a slight bug where you can also edit the label "Public procedures" and "Variables"

2) being able to middle-click (or wheel-click) on a tab to close it... or at the very least including a Close All option under the Window menu, and
Does clicking on the red X not work or something? Or you just want a bigger hit target? (Hmm, I just noticed VS does this... never used that functionality before).
do you mean the x on the tab itself?  If I have a dozen or more scripts open and I want to close them all, I generally want to  hover over one of the tabs, and middle-click rapid fire to close them all. Because the names of the scripts are variable length, the x is not always in the same spot.  But I hadn't thought until now of just using the MDI Child x in the upper right window. So nevermind about the wheel-click request.

Shouldn't it be just the sc/sco files and not the compiled resource package for your source control?
Yup, absolutely. Hindsight's 20/20.


Offline troflip

Re: Recreating complete QFG1 EGA source code
« Reply #22 on: June 14, 2016, 07:59:35 PM »
I can now decompiler QFG2 without error, and recompile all scripts and it appears to run (just walked around a few screens). :D
QFG1 too.
« Last Edit: June 14, 2016, 08:19:14 PM by troflip »
Check out my website: http://icefallgames.com
Groundhog Day Competition

Offline Kawa

Re: Recreating complete QFG1 EGA source code
« Reply #23 on: June 14, 2016, 08:42:04 PM »
Epic.

And you said you didn't want to mess with the decompiler :3

Offline Charles

Re: Recreating complete QFG1 EGA source code
« Reply #24 on: June 15, 2016, 12:16:17 AM »
I can now decompiler QFG2 without error, and recompile all scripts and it appears to run (just walked around a few screens). :D
QFG1 too.
You can now decompile the SmoothLooper function and the --UNKNOWN-PROP-NAME-- in Feature.sc?  That's awesome!

Offline Collector

Re: Recreating complete QFG1 EGA source code
« Reply #25 on: June 15, 2016, 12:43:26 AM »
I just signed up for GitHub. They seem to not want to tell you of the free options until after you sign up. I would really like to establish repos for the decompiled scripts of the official games. As I said I can set them up on my Bitbucket account, which has unlimited repos, and allows them to be private. Its drawback is the number of members is limited to just 5 per account. Do we need more than 5? GitHub has unlimited repos and members, but is higher profile and with no private repos with their free accounts Charles may have a point about needing a less conspicuous repository.

What do others say about where to host the repos?
KQII Remake Pic

Offline troflip

Re: Recreating complete QFG1 EGA source code
« Reply #26 on: June 15, 2016, 02:29:25 AM »
You can now decompile the SmoothLooper function and the --UNKNOWN-PROP-NAME-- in Feature.sc?  That's awesome!

Oops, there's an asterisk beside my claims :P
UNKNOWN-PROP-NAME in canBeHere had to be fixed manually. SmoothLooper is fine though. And I think in both games there were missing scripts that were related to debugging, or development builds. Some of the game scripts reference them, so you have to comment those lines out. But all of that only takes 30 seconds or so. I'm not sure if I can do that automatically. I mean, I probably can, but it might be better to leave those errors in so it's obvious what's "missing". I'm not sure.

I'll probably add a list of tips on decompiling to the documentation that go over the common remaining errors like those and how to fix them/comment them out. Ideally the only errors will be corrupt scripts (like the UNKNOWN-PROP-NAME issue) and missing scripts - i.e. things that are beyond the capability of the decompiler, since they're essentially impossible to fix.

The real big deal is that the decompiler is getting decent enough that - as of now - I don't know of any incorrect code it generates (yes, it still falls back to asm for around 2% of the code). I'm sure there is some, but it's much better than before, where I had to warn people who were modding *just* to recompile the scripts they were changing, not all the scripts - or else something would break.
Check out my website: http://icefallgames.com
Groundhog Day Competition

Offline Collector

Re: Recreating complete QFG1 EGA source code
« Reply #27 on: June 15, 2016, 02:44:48 AM »
Out of curiosity, what version of QfG2 is this?
KQII Remake Pic

Offline OmerMor

Re: Recreating complete QFG1 EGA source code
« Reply #28 on: June 15, 2016, 03:06:17 AM »
Great news Phil!

About the remaining decompilation errors:
I think you should generate special code (inline assembly?) that would tell the compiler to ignore the error and generate as-is.
Same for missing (debug) scripts: the compiler should ignore it. The interpreter tries to load the scripts dynamically, so the user can drop-in the missing scripts and the game would use them.
If you force the user to comment out these references before compilation, this functionality would be gone. You can perhaps make it a compilation warning.

The ideal decompiler/compiler combo would allow to decompile and recompile the whole game and generate identical bytecode with no errors and human intervention.

Offline troflip

Re: Recreating complete QFG1 EGA source code
« Reply #29 on: June 15, 2016, 11:27:16 AM »
Out of curiosity, what version of QfG2 is this?

1.102

About the remaining decompilation errors:
I think you should generate special code (inline assembly?) that would tell the compiler to ignore the error and generate as-is.
Same for missing (debug) scripts: the compiler should ignore it. The interpreter tries to load the scripts dynamically, so the user can drop-in the missing scripts and the game would use them.
If you force the user to comment out these references before compilation, this functionality would be gone. You can perhaps make it a compilation warning.

I agree. I was trying to figure out how this code could be left in without producing an error (and without introducing new syntax). An asm block might be the way.
Check out my website: http://icefallgames.com
Groundhog Day Competition


SMF 2.0.11 | SMF © 2015, Simple Machines
Simple Audio Video Embedder

Page created in 0.191 seconds with 21 queries.