The Dig crashes (3DS / PSP)

[MarioKartFan]

The Dig crashes once Boston begins to walk up the outside of the first column (just after taking his first tram ride and passing by the fossil on the seashore). The crash occurs in the same place on my 3DS and PSP (in both cases the terminal pops up with an error).

[bgK]

What is the error message displayed in the console? (You can take a screenshot using the Luma menu on the 3DS)

[MarioKartFan]

Hi. It is: ERROR: (31:100002:0xA113): Unknown res tag ‘....’ encountered (expected at 22901681+900071 in file dig.la1!

[bgK]

Ok, thank you. I've been able to reproduce the issue and to reduce it to a file read corruption bug:

	char *buffer = (char *)malloc(10*1024);

	FILE *f = fopen("sdmc:/ScummVM/The Dig/DIG.LA1", "r");
	if (!f) {
		printf("Unable to open test file");
		return 1;
	}

	fseek(f, 23808663, SEEK_SET);
	fseek(f, 23808663, SEEK_SET);
	fread(buffer, 1, 2096, f);
	fseek(f, 23810752, SEEK_SET);
	fread(buffer, 1, 4, f);

	buffer[4] = 0;
	printf("1: %s\n", buffer);

	fclose(f);

	f = fopen("sdmc:/ScummVM/The Dig/DIG.LA1", "r");
	if (!f) {
		printf("Unable to open test file");
		return 1;
	}

	fseek(f, 23810752, SEEK_SET);
	fread(buffer, 1, 4, f);

	buffer[4] = 0;
	printf("2: %s\n", buffer);

	fclose(f);

	free(buffer);
	
	return 0;

Which outputs:

1: ����
2: AKOS

While it should output the same thing twice. It's likely a bug in newlib. I'll have to check and report it to them.

[bgK]

Actually it's a bug in a patch devkitPro does to newlib. I've opened an issue upstream: ​https://github.com/devkitPro/newlib/issues/16

As for the PSP side of the issue, coincidentally ScummVM's own implementation of buffered IO has the exact same bug. I have a fix locally. I'll test it some more before pushing it. I'm also thinking of switching the 3DS to ScummVM's buffered IO depending on the response devkitPro does to my issue.

[rsn8887]

bgK, do you think Switch is affected, too? It is compiled using devkita64 which has the same newlib patch bug as devkitpro also, but I am not sure if it is using the (broken) standard ScummVM file routines or the (broken) newlib ones.

[bgK]

The Switch is affected by the bug in devkitPro, but this particular issue in The Dig should not happen. In the toolchain I've used for the 3DS, this newlib commit is included: ​https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;a=commitdiff;h=59362c80e3a02c011fd0ef3d7f07a20098d2a9d5. It fixes a performance problem where a relative seek incorrectly discards the stdio read buffer (a night and day improvement on the 3DS in games like CoMI). On the Switch, the buffer will be discarded and the data will be read from the file instead of from the inconsistent buffer.

[MarioKartFan]

Thanks bgK. When do you think this will be pulled through the latest 3DS development build?

[bgK]

PSP fix: bb276690269a14b44bad7eeaf5a23519ce9390cc (fix ScummVM's buffered IO implementation)
3DS fix: 99b9db456a26ad723689acc2b6cbb9e2bdb8a56b (use ScummVM's buffered IO)

Tomorrow's builds should be good.