Community

SCI Programming => SCI Development Tools => Topic started by: ZvikaZ on September 30, 2020, 08:59:37 AM

Title: SQ3: Where is "Where am I?"?
Post by: ZvikaZ on September 30, 2020, 08:59:37 AM
Hi.

In SQ3 intro, Roger says "Where am I?".
(Which *was*, or *wasn't* heard by the Soundblasters of the Dos era, I don't know - I have seen some different reports of this; I had only PC speaker at the time, anyway...)

It's in room 155. Seems to be sound resource 100:
Code: [Select]
(instance whereSound of Sound
(properties
number 100
priority 5
)
)
ScummVM debugger also reports that it's number 100.

However, I can't play it in SCICompanion.
It seems to treat it just like (an empty) regular MIDI resource, without digital sample.

Citing from http://sci.sierrahelp.com/Documentation/SCISpecifications/19-SoundSubsystem.html#AEN2490 (http://sci.sierrahelp.com/Documentation/SCISpecifications/19-SoundSubsystem.html#AEN2490):
Quote
SCI0 Sound resources can also contain digital samples, although an SCI remake of KQ1 is the only DOS game I know of that includes them. These files still contain MIDI data, but the wave data is appended at the end. The MIDI data is an approximation of the sound effect for hardware that can't play digital sound.

So, is it possible that SCICompanion just ignores the wave data at the end?
Or is there some way to access it?
Title: Re: SQ3: Where is "Where am I?"?
Post by: MusicallyInspired on September 30, 2020, 09:24:07 AM
It worked if you had the right driver but SQ3 was never bundled with that driver. It's available on Sierra Help Pages though.

Usually you have to select the "Digital" patch in the SCI Synth dropdown menu on the Sound Resource (above the play button and transport), but "Digital" isn't an option it seems. SCI Viewer can play it just fine, though.

Digital samples in SCI0 were simply appended to the end of a sound resource, from what NewRisingSun told me. He wrote a program to add sound to an SCI0 sound resource but it had to be 11025 hz 8-bit mono and the resulting sound file could be no larger than a certain byte size (I forget. 65535 I think?).
Title: Re: SQ3: Where is "Where am I?"?
Post by: ZvikaZ on September 30, 2020, 09:35:22 AM
Well, that's interesting info.
Because what I really want, is to modify the sound resource.

So, if SCICompanion can't do it (@Kawa, maybe it can be fixed?), I'd need other tool.
Do you know where can I find NRS tool?
Any other recommendations?
Title: Re: SQ3: Where is "Where am I?"?
Post by: MusicallyInspired on September 30, 2020, 10:04:51 AM
http://sciprogramming.com/community/index.php?topic=592.0

What you're looking for is the AddWAV.exe program.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Kawa on September 30, 2020, 10:19:40 AM
From what I can see, SCI Companion does acknowledge there is a digital sample in there. I open sound.100 and I see one row marked "D". This being a MIDI-oriented editor, it doesn't bother trying to draw the waveform like it does the note events, nor does it try to play anything. Exporting the resource and opening it in Audacity as an unsigned 8-bit 11000 Hz raw PCM, playing that gives me a click of the header, and a perfectly fine "where am I?".

Digital samples in SCI0 were simply appended to the end of a sound resource, from what NewRisingSun told me.

All that being a match, you could make do with a hex editor and some smarts. I'd point out some official tool but I can't seem to find any.

Quote from: Ravi Iyengar
The digital sample header is 44 bytes long. Offset 14 in the header contains the frequency as a short integer. Offset 32 contains the sample length, also as a short integer.
But since we're talking about a digital-only sound, without any MIDI in it, we can ignore all that. Looking at the raw sound.100 in a hex editor, I see frequency 11000 (0x2AF8) at offset 0x33 and the sample length 7415 (0x1CF7) at offset 0x45. Selecting everything from 0x51 (a 0x7A) to the end is a span 7415 bytes in size, so that's our sound All else being equal, one should be able to just remove everything from 0x51 on, record a new sound of the proper format, append it in place of the original, and adjust the size field accordingly.

Attached is a proof of concept where I did exactly that, to make Roger sound like a confused Dutchman.
Title: Re: SQ3: Where is "Where am I?"?
Post by: gumby on September 30, 2020, 08:06:35 PM
Collector and I collaborated a (long) while back and created a sound editor tool for SCI0 sounds.  I can't seem to put my finger on it, but I want to say that it was able to read and create SCI0 sound resources that had both MIDI and WAV data inside them.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on September 30, 2020, 08:32:44 PM
That ended up in a collection of a number of SCI sound utilities with a GUI. Haven't even looked at it in a long time. http://sciwiki.sierrahelp.com//index.php?title=SCI_Sound_Utilities
Title: Re: SQ3: Where is "Where am I?"?
Post by: ZvikaZ on October 01, 2020, 10:27:37 AM
Thanks guys. Sounds great ;-)
I'll check it.
Title: Re: SQ3: Where is "Where am I?"?
Post by: ZvikaZ on October 04, 2020, 11:37:00 AM
It took me time to check, but it works great!
Thanks.

I'd just note, that the GUI has problem with Hebrew file names. I had to change my .wav file to English for it to work (at least, with Windows 10).
I assume that you won't fix it (it's not really worth the bother...), but maybe it should be documented somewhere.
(or maybe add a simple `if` to warn in case of a non-regular-ascii filenames, and suggest renaming)
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on November 29, 2021, 09:33:02 AM
Following this from your link in the Space Quest III French thread, I thought I would look into this. Can you upload the WAV file with the Hebrew file name for me to try?
Title: Re: SQ3: Where is "Where am I?"?
Post by: ZvikaZ on November 29, 2021, 04:42:44 PM
Following this from your link in the Space Quest III French thread, I thought I would look into this. Can you upload the WAV file with the Hebrew file name for me to try?
Well, that's embarrassing. I haven't kept that specific file. And now I cannot reproduce the problem. I have tried Hebrew file name, inside a Hebrew directory, with spaces, and even an apostrophe. But no avail. It always work :-)

So, I'm glad that's your tool is such robust. You can "close" that bug report.
Thanks for wanting to fix it, though ;)

EDIT
====
It is *working*, however, there is something weird about Hebrew files - they don't show Encoding, Sample Rate and Channels information.
(https://imgur.com/a/eVeTzDC)
Screenshot of the difference is attached, as well as the Hebrew file. You can rename it to English, and see the different behaviour.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on November 29, 2021, 08:01:40 PM
The reason for that is that it uses SoX ( http://sox.sourceforge.net/ ) to get the wave file header info. This is the first C# project I did. This is before Phil developed SCI companion 3 and audio support tools for fan SCI projects still relied on the old misc audio SCI tools, including Soundbox. The GUI is reading the file name, but not the SoX utility. To fix it I would have to build SoX to add that support and my C skills are sorely lacking or I would have to roll my own C# class to extract the wav file header info. I am not in too much of a hurry to invest much time on this. I would suspect that SoX would work if it was built with Unicode support.
Title: Re: SQ3: Where is "Where am I?"?
Post by: ZvikaZ on November 30, 2021, 03:18:03 AM
Well, I don't think that it's worth the effort the recompile SoX - it could be a very deep rabbit hole, and my problematic use case is probably rare.

However, if you do want to improve things a little, there are two options that might be easy:
1. If the filename contains non ascii characters, copy it to a temp file with regular English name, and silently work on that temp file. The user won't even be aware of that.
2. If the filename contains non ascii characters, pop up a dialog box recommending to rename it to English in order to have the details table working.

Anyway, this thing is low priority. Your tool is great and does it job perfectly.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on November 30, 2021, 08:13:46 AM
I had thought of your first suggestion, but got distracted by looking into fixing it the right way of getting the info directly in C#. In the end I am not too inclined to invest much time on a tool of such limited usefulness these days, especially since it still works. Any more it might only be of use for translating existing earlier SCI games.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on November 30, 2021, 10:39:22 PM
I decided that simply making a temporary ASCII named copy for SoX wasn't that big of a chore, so here is test build for you to try. If you report that it is OK I'll update the Wiki with the new build.
Title: Re: SQ3: Where is "Where am I?"?
Post by: ZvikaZ on December 01, 2021, 04:30:41 AM
I decided that simply making a temporary ASCII named copy for SoX wasn't that big of a chore, so here is test build for you to try. If you report that it is OK I'll update the Wiki with the new build.
It solved the information table problem - it shows the information for Hebrew files.
However, when I click 'Append', it fails with "Error: String cannot be of zero length. Parameter name: oldValue".
The failure is both for Hebrew and English file names.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on December 01, 2021, 08:04:42 AM
I didn't look beyond the loading of the WAVs. I did change the display in the listview to show just the file name without the path. I'll have to look at the append method to see if it was getting the path to the files from the listview. An easy enough fix.

As I said, this was my first C# project. It was how I first taught myself C#. A lot of the coding is... clumsy. I have never revisited it given that the abilities of Companion have made it mostly obsolete. If there is a need for it perhaps I need to clean it up and redo it.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on December 02, 2021, 04:48:00 PM
OK, give this a try.
Title: Re: SQ3: Where is "Where am I?"?
Post by: ZvikaZ on December 05, 2021, 05:28:02 AM
OK, give this a try.
When I start it, it says: "Unhandled exception" (etc.). If I click "Continue", I get 2 dialog boxes, with "FormMain_GetFocus".
Then the GUI looks OK. Adding Hebrew files seems to work OK, with the correct information table.

However, when I click "Append" button, it writes in the status line "Sound file appeneded with בדיקה.wav" - but the files doesn't appear in the expected directory.
I found it in "C:\Users\Zvika\AppData\Local\Temp", and it's indeed a good file!

So, bottom line, 2 problems are left:
- The unhandled exception and the strange dialog boxes upon startup
- Copy back the resulting sound file from temp directory to original directory

Thanks
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on December 05, 2021, 09:38:31 AM
It looks like the actual AddWav.exe program does not like Unicode either, so I will have to deal with non-ASCII file names completely via the GUI. It uses SoX not just for extracting the header info of the WAV files, but also for downsampling them to prepare them for AddWav. Gumby updated a few of these DOS tools to Win32, but I don't think he did AddWav. If this GUI and the tools it uses are useful to translations I might think about redoing it to fix where Unicode breaks it if you are willing to test it for me.

Anyway, here an updated version. It will now output to the same folder as the original WAV. That particular error code does not seem to break it, so I set the error reporting to ignore that warning. Could you try a resulting embedded SOUND resource in an actual SCI game to be sure that it is producing a good SOUND resource?
Title: Re: SQ3: Where is "Where am I?"?
Post by: ZvikaZ on December 05, 2021, 12:02:30 PM
It looks like the actual AddWav.exe program does not like Unicode either, so I will have to deal with non-ASCII file names completely via the GUI. It uses SoX not just for extracting the header info of the WAV files, but also for downsampling them to prepare them for AddWav. Gumby updated a few of these DOS tools to Win32, but I don't think he did AddWav. If this GUI and the tools it uses are useful to translations I might think about redoing it to fix where Unicode breaks it if you are willing to test it for me.

Anyway, here an updated version. It will now output to the same folder as the original WAV. That particular error code does not seem to break it, so I set the error reporting to ignore that warning. Could you try a resulting embedded SOUND resource in an actual SCI game to be sure that it is producing a good SOUND resource?

This one is better. However, when creating the sound file, it tries to create it in "C:\Program Files (x86)\SCI Sound Utilities", and fails because Windows denies access to that path.
You can see this one for yourself, it fails also with English file names.

Anyway, I'm not really sure that fixing the Hebrew file names issue is such important :-)
If you want to continue, I'm happy to test your fixes. But if it wastes your time, well, a simple warning message can do as well...
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on December 05, 2021, 09:26:41 PM
That is strange, it should be outputting to the same folder as the original WAV file. It is not doing that on my machine. I am running it out of VS, not an installed copy. You can try deleting the Registry key it created to see if that helps. One of the tings I am doing is converting the settings over to XML instead of the Registry.

HKEY_CURRENT_USER\SOFTWARE\SCI Sound Utilities

Also, I take it that you are using the tool from the main form, not the secondary AddWav form. I have not touched that one, yet.

Now that I have started, I have been more or less refactoring it to fix the silly newby errors and using more "best practices". Hebrew would not be the only thing that it would be good for it be Unicode. Any European language that uses diacritics might choke on it.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on December 05, 2021, 09:51:20 PM
As I noted, Gumby ported some of these tools to Win32. I can't find any source for AddWav. Gumby, if you see this did you have the source to it? Did you just reverse engineer it for your ExtractWav?
Title: Re: SQ3: Where is "Where am I?"?
Post by: MusicallyInspired on December 05, 2021, 10:10:23 PM
There's no source for AddWave. NRS didn't provide any.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on December 06, 2021, 10:37:15 AM
I had forgotten that NRS wrote it. I don't think he has ever released any source from his projects. As I noted, Gumby did write ExtractWav, which does the reverse. Not sure how hard it would be to recreate AddWav to support Unicode, but probably beyond my abilities.
Title: Re: SQ3: Where is "Where am I?"?
Post by: gumby on December 06, 2021, 07:15:16 PM
Yeah, I don't have any source for AddWave.  I've attached a zip file of the executables and C# sources that I created nearly a decade ago... hopefully it'll help.  Collector, I think you are right that I must have just created the AddWave functionality in the SCI Sound Utilities application by using/inspecting the ExtractWave functionality.
Title: Re: SQ3: Where is "Where am I?"?
Post by: MusicallyInspired on December 06, 2021, 08:32:46 PM
According to what NRS told me way back when, it's "basically" just amending audio data onto the end of the file. But I'm sure it has to change some header information too.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on December 06, 2021, 08:56:24 PM
I already have your source for ExtractWave. Feel like taking it on to recreate AddWave with Unicode support?
Title: Re: SQ3: Where is "Where am I?"?
Post by: Kawa on December 07, 2021, 06:05:21 AM
* appending
Title: Re: SQ3: Where is "Where am I?"?
Post by: gumby on December 07, 2021, 09:32:46 AM
I'll take a look at it this weekend.  My only hesitation is around the LZW encoding.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Kawa on December 07, 2021, 04:51:00 PM
Last I checked LZW encoding compression only applied to the resources when stored in a resource volume, not to loose files?
Title: Re: SQ3: Where is "Where am I?"?
Post by: gumby on December 08, 2021, 09:25:09 AM
Hrm.  I wonder why I needed it for ExtractWave - maybe that utility pulls data out of a resource file instead of a patch file.  I'll have to take a closer look.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Kawa on December 08, 2021, 03:33:23 PM
...or really any compression at all, besides some basic RLE on views and I think non-vector background images.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on December 18, 2021, 02:42:56 PM
Hey Gumby, have you made any progress? I have been cleaning up the old code for the GUI to make it easier to make any needed changes, so no rush. Just curious if you have had a chance to look into it more.
Title: Re: SQ3: Where is "Where am I?"?
Post by: gumby on December 21, 2021, 10:34:11 AM
No, not yet. When I find some free time I'll look more into it.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on December 21, 2021, 04:32:45 PM
I understand all to well about the lack of time. I am mostly finished with cleaning up the old code.It no longer uses the Registry to store settings. All settings are now XML based. I fixed some ambiguous variables with all camel/lower casings. I've weeded out most of the VB garbage that was a carry over from the original porting from starting it in VB.net. Some awkward methods with a more straight forward ones. It is much more readable code, now and I am ready to make what modifications are needed to build in Unicode support.

I guess I'll shelve it for the time being until You have had a chance to determine if you are willing or can take it on before I implement any kludgy workarounds. Even so, I'll still need to use workarounds for no Unicode support in SoX.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Kawa on December 22, 2021, 08:57:26 AM
I am ready to make what modifications are needed to build in Unicode support.
If it's .Net, you basically already have Unicode support.
Title: Re: SQ3: Where is "Where am I?"?
Post by: Collector on December 22, 2021, 09:04:46 PM
But AddWav is not. The program is mostly just a GUI for existing programs. All I can do with the NRS version is to come up with workarounds such as copying/renaming Unicode files to ASCII names to use with AddWav and then renaming them back to the Unicode name. I can do that, but would prefer a version of AddWav that I could throw any file at it without resorting to such workarounds.