GRIM: MD5 dialog can’t be cancelled in Grim Fandango

[ccawley2011]

The MD5 dialog in Grim doesn’t have a cancel button, so it’s necessary to wait until all files are checked before ScummVM can be interacted with. It also doesn’t appear to poll for events, which means that on a cooperatively multitasked operating system like RISC OS it’s not possible to interact with anything outside of ScummVM either.

[deckarep]

Hi ScummVM Team,

I was looking at this bug to see if I can contribute a patch to improve it and I discovered that one of the reasons the md5 detection code for the GRIM engine is slow is that it's doing an md5 checksum of the full file (all bytes).

However,

It seems like somewhere Grim is supposed to check only 5k bytes per this function:

./scummvm --md5 --md5-path=<file> --md5-engine=grim

Notice the output says:
VOX0001.LAB: <hash> (first 5000 bytes), <size> bytes

Since Grim has a lot of files with some being quite large to check for it might be good to modify this related code to cap all md5 checks at 5k bytes to speed up this process for end users.

I'm not sure why there's an inconsistency here but there is a potential for this md5 integrity check code to be much faster.

[sev-]

Would be nice to get fixed before 2.8.0 release

[somaen]

Grim intentionally checks the entire file of all the files, as we had a lot of issues with bugs filed from scratched CDs. That said, we should fix the UX issues here.

[aquadran]

MD5 check use common MD5 checksum code which read whole file at once, there is no any possible polling while reading file.
This is limitation of MD5 code, not engine specific issue.
Also its performance issue depends on speed access to read files.

[sev-]

This could be implemented if we make our md5 checksum more granular.

I see two approaches:

1. Pass a callback and number of bytes or milliseconds to call in-between. This might not truly work in GUI, though due to race condition on screen updating
2. Implement state for md5, so it is re-entrant, something like

  md5.setup(data, size, milliseconds);

  while (md5.process()) {
     update_gui();
  }

  checksum = md5.getResult();