SCI: GK2: Improve bad AV sync in GK2 VMDs

[csnover]

At least GK2 Chapter 6 intro video has some lip sync problems. This is a source video problem and not an engine bug—GK2 1.1 comes with many re-encoded videos with fixed sync. Investigation is necessary to determine whether there is a way to fix the videos from GK2 1.0.

[m-kiewitz]

Where exactly do you experience those lip-sync issues?

I tried both of my GK2 versions and both look and sound fine (on Windows XP, warped to chapter 6 via debug menu).

This here is a Let's Play video of chapter 6 intro:
​https://www.youtube.com/watch?v=DdyiJlYw7HA

[csnover]

Just to keep this ticket up-to-date, it has been confirmed that this is an engine bug affecting at least GK2 and Phant1 VMDs. (FFmpeg’s VMD decoder is also affected.)

1. It seems there is a mismatch in the number of video frames and audio frames in some of these files, which causes drift over time? (GK2 7180.VMD)
2. There are also short videos with silent audio blocks that are out-of-sync (GK2 7510.VMD), which seem to be improved by changing the handling of silent blocks

In any case, changing the VMD code in ScummVM breaks Urban Runner videos, so it seems that SCI is using some derivative VMD decoder and probably needs to be RE’d to find the differences.

[csnover]

As it turns out, not quite as “confirmed” as I initially thought. (Note to self, don’t rely on Let’s Play videos as an exclusive source to determine bugginess!)

The 1.1 version of GK2 from GOG has mostly (~80%) different VMDs than the 1.00 release, and these videos do not exhibit any of the serious AV sync problems seen in ffmpeg with the 1.00 videos. So it seems that the assets in the original release of GK2 are, in fact, broken, and were re-encoded for future releases.

Comparative analysis of some known bad & known good videos (bad videos are 1.00):

7180.VMD header differences

field	1.00	1.1
video codec	0x7F24	0x7724
sound slice size	0xF76A	0xF763
sound slice count	0xE	0x1D
frame table offset	0x33727C6	0x3372672

20.VMD header differences

field	1.00	1.1
audio sample rate	22050	22222

Phant1 videos don’t actually seem to be affected in this way; it looks like there were probably some unrelated issues with the VMD playback code affecting Phant1 that have since been fixed.

Now, speculations on what might be used to fix the 1.00 videos:

The different audio rate in 1.1 20.VMD makes me wonder if perhaps the audio sample rate of these bad 1.00 videos isn’t right, and should be overridden to ​22222Hz or 22157Hz (a magic number I found in some of the VMD code). I am not sure what the ramifications for the decoder would be if this is the problem since I would imagine that there would not be enough audio data in the frame data to keep up, and we would need to steal audio from future frames. I guess we’ll find out…

[csnover]

This is a problem for some version 1.1 videos as well, like 1930.VMD, which was not re-encoded.

[sev-]

Closing it as due to the broken game assets.