SCUMM/HE: PJGAMES: Add audio modding support for Pajama Sam: Games to Play on Any Day

[SomeRandomHEFan]

Over the past months, I've worked on updating the Bandcamp releases of various Humongous Entertainment soundtracks to support playing them in their corresponding games via ScummVM's audio modding feature.

I'd like to update Pajama Sam 3's soundtrack again so that it can also be played in Pajama Sam: Games to Play on Any Day. Most of that game's music is from the same soundtrack. However, this is currently not possible, as ScummVM lacks audio modding support for it. The game was made after Humongous Entertainment made the jump to 22050 Hz, so it probably handles audio rather differently.

It would be nice if the scope of ScummVM's audio modding feature could be expanded to cover this game.

[AndywinXp]

Hi! I'm currently rewriting the audio system for all the HE games and I'm expecting to finish the job in about a month (well, hopefully earlier). Since this rewrite is done from source code and disassemblies, it's going to be accurate and it's going to automatically nuke several bug tickets that we currently have on the tracker.

Of course this comes at a cost: I'll have to re-implement the audio modding function for the whole subsystem (twice actually: one for the software mixer used by most of the games, and one for the Miles Sound System driver emulation which powers up this game and the later ones).

Do you have any technical details about the format of the modded audio files? Are they repackaged in some manner? I do not have any of them on my hands currently, but I'd like to know more about it. Let's keep in touch either on this comment section or on the ScummVM discord server.

[SomeRandomHEFan]

Good to hear that progress is being made in that department. I'd appreciate if this resulted in the addition of audio modding support for this game.

The files are not packaged in any way. My implementations involve the music files and sound effects being in a "sound" directory and having names corresponding to their internal IDs. Voice clips are in a "speech" directory and have names that I believe correspond to their location in the .HE2 file. However, there are other ways to make ScummVM accept files. IIRC, you can also put them in the game data's root directory and give them names like "soundXXXX" or "speechXXXX." Not quite sure I got all the details right, though.

ScummVM should accept any files in the formats FLAC, WAV, MP3 and OGG, I believe. The specific files used in the Bandcamp releases should be in either mono or stereo, have sample rates of either 44100 Hz or 48000 Hz and a bit depth of either 16 or 24. Some have plenty of metadata, including embedded album images, while others don't.

Does that information suffice? Let me know if I can offer any more help.

Is there any chance that your work could also add support for replacing the MIDI music in HE's earliest releases, as explained at #14266?

Also, is #7150 one of the bugs you expect to be fixed through this? That would be great.

[AndywinXp]

Replying to SomeRandomHEFan:

Good to hear that progress is being made in that department. I'd appreciate if this resulted in the addition of audio modding support for this game.

The files are not packaged in any way. My implementations involve the music files and sound effects being in a "sound" directory and having names corresponding to their internal IDs. Voice clips are in a "speech" directory and have names that I believe correspond to their location in the .HE2 file. However, there are other ways to make ScummVM accept files. IIRC, you can also put them in the game data's root directory and give them names like "soundXXXX" or "speechXXXX." Not quite sure I got all the details right, though.

ScummVM should accept any files in the formats FLAC, WAV, MP3 and OGG, I believe. The specific files used in the Bandcamp releases should be in either mono or stereo, have sample rates of either 44100 Hz or 48000 Hz and a bit depth of either 16 or 24. Some have plenty of metadata, including embedded album images, while others don't.

Does that information suffice? Let me know if I can offer any more help.

Thanks, for now this will suffice; because of the highly variable sample rates and because of the multiple audio compression types, I'll probably have to generate a dedicated code path for these audio files to play, which skips the original the software mixer. I'll see what I can do.

Thanks, I'm sure I will have more questions as I go forward.

Is there any chance that your work could also add support for replacing the MIDI music in HE's earliest releases, as explained at #14266?

It's not impossible, I'll see what I can do!

Also, is #7150 one of the bugs you expect to be fixed through this? That would be great.

Absolutely; the rule of thumb is: if you can hear something on the original exe, that's what you will hear on the reimplementation as well.

[SomeRandomHEFan]

I just remembered that there may be one potential obstacle with making the MIDI tracks moddable. Early on in Putt-Putt Goes to the Moon, there's a part that I suspect may use some more complicated trickery in the MIDI versions than in the versions with streamed music.

There's a music track that was split into two separate tracks in the streamed versions, with the first part covering when the bridge collapses and Putt-Putt falls into the moon goo and the second part then looping endlessly when he's stuck down there. However, I just examined the debug log, and it looks like the DOS version treats both parts as the same track (the ID is 79). I suspect that it loops in a special way in order to prevent the first part from being replayed.

So it might not be easy to make the audio modding feature support something like that. Ideally, I think it should expect separate files for such non-standard loops.

[AndywinXp]

Ouch, that indeed makes things a lot more complicated

[SomeRandomHEFan]

I see your work recently got merged. But it seems there's still no audio modding support for this game. Is that something you plan to add in the future?

[AndywinXp]

Hi! Yes, I'm planning that for the immediate future, but I need to have some files to test it on. Mind to share one or two sample tracks for this game in private so I can get it to work asap? :)

[SomeRandomHEFan]

Good to hear. I just asked Timothy Knox (the person responsible for the soundtrack remasters) if I can share a few of the tracks.

Do you have a public email address I can send them to?

[AndywinXp]

Thanks! I do not have a public email address, but feel free to barge into the ScummVM discord server, then into #engine-scumm and ask for Bosca (which is me :p)

[AndywinXp]

Heyhey, I'm almost done, I just have to add the feature to the detection entry; is there any other game which previously didn't have support for audio modding that you are planning on supporting in the near future? So I can edit the detection entry for that one as well

[SomeRandomHEFan]

I currently don't have any plans to work on ScummVM packs for any other HE games that use the Miles Sound System. To the best of my knowledge, those never had higher quality versions of their soundtracks released.

But I think the option to mod their audio should be made available regardless, just in case a use for it ever arises. Is there anything that speaks against enabling support for all games that can theoretically handle it?

Or am I misunderstanding what you're trying to do?

[AndywinXp]

You understood correctly; well, I guess it wouldn't hurt to allow audio modding for every game... anyway, thanks for the assistance, this feature request is now complete!

[SomeRandomHEFan]

Great to hear. I'll probably get started on a ScummVM pack relatively soon.

I take it you don't plan to try making the MIDI tracks moddable as well? Not that I'd blame you for giving up on that, given that it may be challenging to find a solution for the customized looping.

Do you have any idea how the original versions of Putt-Putt Goes to the Moon pull that off? If the MIDI file itself has information on how it should loop, perhaps ScummVM could optionally allow for all loops of a track after the first to be overridden with a different file? But if the looping is controlled by a script, I could indeed see it being nasty to interfere with that.

[AndywinXp]

Heh, good question. The fact that MIDI files can jump anywhere on the track poses a challenge which I don't know if I'm able to face right now. I guess we'll have to see later down the line... Right now the objective is to have this sound engine as stable as possible; non-trivial new features for now are not excluded, but are not planned for the immediate future.

[SomeRandomHEFan]

Sure, I can respect that. There are obviously more important matters to deal with.

In case you ever decide to give it a shot, I just did some investigating myself. I managed to extract the track in question by using ScummEX and then deleting the first 40 bytes. The result is a playable .xmi file. It loops just like in the game. So that means the non-standard loop is recorded in the file itself and not controlled by the game's scripts.

I'm hardly an expert on this matter, but perhaps a functional audio modding implementation could expect each track to only have one replacement by default, but optionally use a second one for subsequent loops. That's to say, if the game wants to endlessly play track 79, it could load a file 79.flac. If that's the only file that exists, then it will be replayed for all additional loops. But if a file 79loop.flac also exists, then that will be used for all subsequent loops instead. Does that sound realistic?

[SomeRandomHEFan]

I just set things up so that my copy of Pajama Sam: Games to Play on Any Day can play remastered versions of most of its music. But when testing it, I noticed some strange anomalies. The first seconds of certain tracks get cut off.

My experiments indicate it's not caused by the files themselves, but their track IDs. It seems to happen with any file that I use to override tracks 8007, 8008, 8010 or 8015. There may be more. It's hard to test, since the game plays tracks 8001-8016 at random.

So I think something isn't quite working correctly with your audio modding implementation for the game.

[AndywinXp]

Thanks, I'll take a look after my vacation.

[AndywinXp]

Here I am. I have been testing this issue for the last half hour but I haven't been able to reproduce it whatsoever. May I ask for some more details on how to do it? E.g. are the files which you are using in a certain format?

[SomeRandomHEFan]

Okay, then try this. Create a "sound" directory in the game's data and put 16 copies of one of the files I sent you (I used the first one for my test just now) in there with the names 8001-8016.flac. Then launch ScummVM in a debug level high enough for the console to show track names. Launch the game and mash ESC until you're in the main menu and music starts playing. Most of the times, the music will play normally, but if the beginning is skipped, check the track ID. From my experience, it always happens with IDs 8004, 8007, 8008, 8010 and 8015. Furthermore, the game will skip a different amount of audio for each ID.

[AndywinXp]

Thanks, it appears I have fixed it, you should be able to verify that on tomorrow's build.

[SomeRandomHEFan]

Okay, all tracks play correctly with the new build. Thanks again. Now I can whip up a ScummVM pack for the game.