Possible new version of SCI Companion

[Collector]

I was going to focus on SCI1.1, since I figured the more advanced the interpreter the better. But that's a good point, there is no internal debugger for any SCI1.1 games, right? That sucks.

That's actually going to make it really hard for me to validate my decompiler... I suspect I'll run into random crashes with no way to know what happened. I guess I'll have to test the decompiler mainly on SCI0 games, and just hope there aren't too many SCI1.1-specific issues.

Does SCUMMVM have a debugger? I'm pretty sure FreeSCI did, not sure if it made it through the move to SCUMMVM.

Looking to SCI1.1 makes sense to me as well. ascetically there is little difference but having the audio capabilities of SCI1.1 is a definite plus. ScummVM does have an internal debugger, but it is specific to ScummVM. I don't see how it could help, outside of using it to run your game, but the hash (usually that of the RESOURCE.MAP) needs to be entered into the SCI detection tables in the source, which would change every time a resource was added.

[troflip]

ScummVM does have an internal debugger, but it is specific to ScummVM. I don't see how it could help, outside of using it to run your game, but the hash (usually that of the RESOURCE.MAP) needs to be entered into the SCI detection tables in the source, which would change every time a resource was added.

Why wouldn't it help? Presumably something that makes Sierra's interpreter crash would also make ScummVM's interpreter crash, or throw up some warning or error at least. I can't be in the situation of having to validate my SCI1.1 generated code without a debugger to find out exactly where things crashed.

ScummVM should detect SCI versions based on the types and format of resources it finds. I've looked over its code quite a bit when implementing my own version checking. I assume the hash is to identify specific games for special workarounds? Or are you saying it will block any fanmade games that don't have some hash that matches their resource.map or something?

[edit:]
Well, the debugger looks to be fine, I tried stepping through some SQ5 byte code.

But SCUMMVM doesn't seem to want to play the SCI Studio template game, it just crashes. It plays some of the fan-made games I tried, but it looks like it already knows about them. How does the hash thing work? It seems like there should be a way to bypass it for development. Playing a game in SCUMMVM is a lot nicer than clunky DosBOX.

[MusicallyInspired]

Ugh no. I dislike ScummVM. Especially for SCI1.1 games. It doesn't get cycles or speed or something right which causes a lot of animations to play as fast as they can. Take the SQ5 intro sequence when Roger is exiting the bridge simulator. Those rats aren't supposed to run away that quickly. Try it in DOSBox and lower the CPU cycles to see what I mean. SCI1 games, on the other hand, don't suffer from this issue in ScummVM. Animations play perfectly, more or less. Another issue is that it will not play MIDI sounds if there is a digital equivalent so you're always stuck with them, even in cases where the digital files are just lo-fi versions of recorded MT-32 sounds. The better sounds are actually not digital in many cases. That problem extends right back to SCI0. Even though there's an option in the ScummVM game-context menu to disable them, it doesn't work. ScummVM is good in a pinch, but it is not the goto solution for best quality.

[lskovlun]

Does SCUMMVM have a debugger? I'm pretty sure FreeSCI did, not sure if it made it through the move to SCUMMVM.

It does, and it's better than anything Sierra had. 

With a little work, it would be possible to make it work on the source level as well.

[lskovlun]

Ugh no. I dislike ScummVM. Especially for SCI1.1 games. It doesn't get cycles or speed or something right which causes a lot of animations to play as fast as they can.

You know what? That's because Sierra's programmers deleted the call to Wait() from the main loop - the result being, as you note, that SCI1.1 games run as fast as they can on any processor on any operating system. We work around it by putting the delay loop in the GameIsRestarting() call instead, but there are cases where that is not enough.

When the dev environment for SCI1.1 comes out, you're free to reinsert the Wait() call (the kernel routine still exists). There may be complications that I don't know about, but we can work on those.

[MusicallyInspired]

That's illuminating and exciting. Still would rather use DOSBox, though. Obviously most others wouldn't, though. And that wouldn't stop me from insuring ScummVM compatibility should there ever be differences, let me be clear.

[troflip]

Haha, fair enough, I haven't used ScummVM much, I didn't notice the mice thing. If DOSBox, "gets it right", presumably it wouldn't be too difficult to have Scumm get it right too though?

After trying DOSBox again after all these years, I was surprised it really wasn't much better than it used to be. It still "traps" my mouse inside the window. And it sometimes processes keystrokes when the window doesn't have focus... so if I switch back to DOSBox, suddenly the game goes crazy popping up messages boxes and such. It just feels... "unfinished". This makes it pretty much useless if you're switching back and forth between DOSBox and other apps (like you would be if you were developing a game).

[MusicallyInspired]

Huh. I've never run into that keystroke problem. I don't mind the captured mouse thing. Especially for adventure games as it's really annoying to try and hit the very far edge of the screen and then having the mouse go too far. Ctrl+F10 releases the mouse, but then you don't control the mouse in the game until you hit it again. ScummVM does offer both modes so it's got that going for it. I've considered DOSBox finished and polished for a while now. At least for my needs.

ScummVM, on the other hand, still feels unfinished to me. Mostly only in the SCI department. I still consider DOSBox the defacto standard because as good and impressive as ScummVM's reverse engineering is for SCI, it's still not as good as the original interpreter.

[Collector]

After trying DOSBox again after all these years, I was surprised it really wasn't much better than it used to be. It still "traps" my mouse inside the window. And it sometimes processes keystrokes when the window doesn't have focus... so if I switch back to DOSBox, suddenly the game goes crazy popping up messages boxes and such. It just feels... "unfinished". This makes it pretty much useless if you're switching back and forth between DOSBox and other apps (like you would be if you were developing a game).

The mouse capture is easy. Under the SDL section of the conf set autolock to false. There are only a couple of times where it may be needed. When an event is is only triggered at the very edge of the screen and for VESA games where the cursor does not always snap to the point of entry of the mouse within the window. As MI noted, you can also do it from the keyboard with Ctrl+F9 and Ctrl+F10. Once you learn what you can do from the conf, using DOSBox becomes much easier.

I have never noticed the keyboard input problem, either. Perhaps you have not made the other window the active one? Once DOSBox is not the active window that should never happen. You can see similar problems with inactive windows in other programs. Even Office will not accept input on first click. The first click only makes it the active window.

In any case, nearly all of the complaints that I have heard about DOSBox are easily solvable or avoidable by reading and becoming familiar with the conf and makes using DOSBox a snap. Once you get it sorted out you generally get much better results than with ScummVM and DOSBox's compatibility with DOS games is quite high.

As to the development of DOSBox, it is true that the official 0.74 is getting a bit long in the tooth, but development has not slowed. The latest SVN has many improvements. Bug fixes are frequently dealt with in the beta testers forum, even if at this point those bugs are mostly from obscure games that used undocumented and unusual aspects of DOS. Then there are the unofficial builds that hold a lot of promise, like the x branch that is adding features like proper PCI emulation.

I too have issues with ScummVM, especially when it comes to SCI, not only because of lack of support for anything later than SCI1.1. I can generally get much better sound from DOSBox. AdLib emulation in ScummVM offers no advantage since the code was lifted directly from DOSBox. While SVM does have Munt built in, it is easy enough to use the Munt driver or QT app with DOSBox. There is also the daum build of DOSBox with Munt built in. Any other thing that some would cite about SVM's sound is easily had in DOSBox, too. FluidSmith and such can be used with, even if not built into DOSBox. This also has the advantage of have the latest with DOSBox while ScummVM often suffers from an ancient version of these extras. Munt does not get updated with every new build of SVM.

Another beef that I have with SVM is that they have been heavy handed with AGI and SCI. The initial implementation of SCI0 forced "undithering" on its users, since they knew better about SCI than everyone else. They also tried to usurp this community to divert game development to their forums and then advise people to use modern game development systems instead. I was even scolded for putting the Free SCI specifications document on our Wiki.

[troflip]

I guess maybe I'm just unlucky with Dosbox bugs.

If I try to bring up the briefcase menu in SQ5, and tab to another window on my desktop, then back, it will often go crazy, like the tab key is being pressed non-stop (until I hit escape). So alt-tabbing between DosBox and another window is basically broken.

Thanks for the autolock tip, that makes things a little better. autolock false should really be the default, IMO. Users shouldn't have to mess with config files.

[lskovlun]

Just tried Sierra SCI under FreeDOS in VirtualBox. It has exactly the same speed issues.

[MusicallyInspired]

Well of course it would. It's a system speed issue, as you said. DOSBox's cycles settings makes this a non-issue. I was always confused why in ScummVM SCI1 games behaved properly and SCI1.1 didn't. Hearing that it's a SCI scripting issue makes more sense. Even so, DOSBox doesn't have this issue...

Also, Collector, CTRL+F9 quits DOSBox. CTRL+F10 is a toggle.

I was even scolded for putting the Free SCI specifications document on our Wiki.

To be fair, they did say it was out of date. And I wouldn't say there was any scolding. Not in that thread, anyway. I specifically recall the phrase "please consider using our SCI specifications" or something to that effect because there were numerous corrections of inaccurate information. But yeah, the undithering and attempted usurpation were not shining moments. At least they backpedaled on the undithering stuff.

[Collector]

Thanks for the autolock tip, that makes things a little better. autolock false should really be the default, IMO. Users shouldn't have to mess with config files.

Agreed. That is what I have set in my main conf file since there are far more games it is not needed than do. I was forgetting the autolock hot key was simply a toggle. I just always do it through the conf.

And MD5 was unhappy with our Wiki having the specifications doc, insisting there should be no other doc than their own. I will grant you that others (sev?) did step in to say to him that that is just the nature of the internet and also acknowledged that this community did not have access to edit their Wiki, so it was not all of them.

[lskovlun]

But yeah, the undithering and attempted usurpation were not shining moments. At least they backpedaled on the undithering stuff.

You do know that Matt (the FreeSCI testing guy) put together some interesting color combinations for SQ3 using the palette language we developed?
I was going to improve on it for a course on DSLs I attended, but I had to abandon it for time reasons.
 Myself, I quite liked the results of Christoph's unditheritng algorithm.. I guess it's a matter of taste, and it was always a configuration option.

[MusicallyInspired]

Actually, I have absolutely nothing against the undithering feature (other than preferring the original dithered look personally). I was excited to see how it could develop further and be closer to how the original dithering was perceived. I admired him for it! I and others would never have spoken against it at all in the first place had things happened differently. What turned me off was the enforcing of it to be the default option and the whole pompous attitude he had about it, claiming that it was exactly what Sierra, the artists, and everyone wanted and then proceeding to belittle me and avoid my arguments against enforcing it as the default option when I actually made valid points and twisted everything I said. Which just made the debate continue in circles needlessly with my trying to correct his misconceptions of my opinions and getting the subject back on track. The man was arrogant about it to say the least. To this day the undithering checkbox is not and undithering checkbox at all but a dithering checkbox which is enabled to "add" dithering, like it was never there to begin with. Just more evidence of that attitude. (I am apparently blind or just plain not observant because this is apparently not true anymore so I'm redacting my statement) I understand the technicality that technically the dithered colours are their own values and could be represented as anything, but his assertion that those colours generated by his algorithm were actually something like what the artists were seeing when they drew these backgrounds and that Sierra created drivers that translated that down into the standard dithered 16-colour palette was just plain asinine.

Anyway, that's five years ago. Over and done with. I don't want to dredge it up again here. Like I said, I was looking forward to seeing how it could develop. Perhaps even to a point of enhancement where there could be some advanced combining of colours to increase the palette count to more than 256 colours by continuing to average colours and making more gradual changes instead of the flat look it currently has. The backpedalling I was referring to was the decision to change it from opt-out to opt-in.

The custom palette combinations certainly sound interesting. I'm sorry to hear it didn't get developed further. Actually, it seems like all work on the SCI side of things just stopped cold suddenly for some reason.