COMMON: macresman.cpp (i think) interferes with "Mass Add"

[raziel-]

ScummVM 2.9.0git (Mar 11 2024 21:02:00)
Using SDL backend with SDL 2.30.1
Features compiled in: Vorbis MP3 RGB zLib MPEG2 MikMod Theora VPX AAC A/52 FreeType2 FriBiDi JPEG PNG GIF cloud (servers, local) ENet SDL2 TinyGL OpenGL (with shaders)

When i let a "Mass Add" run over my Games: partition i get a lot of instances where a requester window comes up, asking me to insert volume
_(underscore)_(underscore)MACOSX/Games:
while searching for gamefiles (see screenshot).

I don't have such a subdirectory in my partition.
The only instance of that name, i could find, is in common/macresman.cpp on line 188

I don't have the slightest idea how to suppress that.
It takes about a minute for the OS requester to go away by itself or hundreds of clicks.

I could turn those requesters off for my platform solely (just like it's done in
backends/platforms/sdl/amigaos/amigaos-main.cpp, lines 59 and 66)
but i wouldn't know what kind of impact or intrusion that would bring or if it's even possible, since it would a platform command(?)

Please advise

AmigAOS4 - BE - SDL - PPC
gcc (adtools build 11.3.0) 11.3.0

[digitall]

@raziel- : You compile from scratch? If so, are you enabling all engines? Just be aware that the Mac Resource Manager is used by various engines in detection of resources. So you may want to try disabling a few of the engines which use this and see if the mass add stops being noisy? I would suspect this may be the DIRECTOR engine... so try ./configure --disable-engine=director && make clean && make

The other engines which use Macresman include pegasus, hadesch, groovie, wage, kyra, scumm, sci, mohawk, saga and composer. You may want to try cutting these down and seeing which is the main cause of this...

[raziel-]

@digitall

yes, and yes

will do when i get back home

thank you

[raziel-]

@digitall

i can confirm that the director engine is the noisy one.

all the other engines you listed are fine, no windows coming up, doing what they should in an automated fashion with Mass Add.

something director engine does that the others avoid?
anything i can do to fix it?

thank you

[sev-]

This is definitely related to your backend.

What we are trying to do is call Common::FSNode::exists(). E.g. a normal check for a file existence. This happens in hundreds of other places for other reasons, so should be fixed by the backend.

[raziel-]

@sev-

Ok, please bear with me as i need to understand what is happening where (not that i'd be able to fix it, but information and stuff)

in common/macresman.cpp, line 170++

SeekableReadStream *MacResManager::openAppleDoubleWithAppleOrOSXNaming(Archive& archive, const Path &fileName) {
	SeekableReadStream *stream = archive.createReadStreamForMember(constructAppleDoubleName(fileName));
	if (stream)
		return stream;

	const ArchiveMemberPtr archiveMember = archive.getMember(fileName);
        const Common::FSNode *plainFsNode = dynamic_cast<const Common::FSNode *>(archiveMember.get());

	// Try finding __MACOSX
	Common::StringArray components = (plainFsNode ? plainFsNode->getPath() : fileName).splitComponents();
	if (components.empty() || components[components.size() - 1].empty())
		return nullptr;
	for (int i = components.size() - 1; i >= 0; i--) {
		Common::StringArray newComponents;
		int j;
		for (j = 0; j < i; j++)
			newComponents.push_back(components[j]);
		newComponents.push_back("__MACOSX");
		for (; j < (int) components.size() - 1; j++)
			newComponents.push_back(components[j]);
		newComponents.push_back("._" + components[(int) components.size() - 1]);

		Common::Path newPath = Common::Path::joinComponents(newComponents);
		stream = archive.createReadStreamForMember(newPath);

		if (!stream) {
			Common::FSNode fsn(newPath);
			if (fsn.exists()) {
				stream = fsn.createReadStream();
			}
		}

		if (stream) {
			bool appleDouble = (stream->readUint32BE() == 0x00051607);
			stream->seek(0);

			if (appleDouble) {
				return stream;
			}
		}
		delete stream;
	}

	return nullptr;
}

This is the only place in the code where __MACOSX is used to look for something...and being "pushed back" (to the current path, i guess?)

While it *does* work, since it *is* added * but in front* of my path, like
__MACOSX/Games:, which obviously doesn't exist (Games: is a partition).

Iiuc (and i sure don't) that is part of a path that should probably go *after* the main search path?
Or even at the end of the current path that is used to look for game files?

But it doesn't.
How do i make that partial path appear at then end of my current search path?

NB:
I have absolutely no idea how MacOS works...is that __MACOSX some special directory where games place their data files?

Please help

Thank you very much

[sev-]

Cross-referencing #14843

[sev-]

Yes, I see the problem now and it is also mentioned in #14843. E.g. the code is buggy and is trying to look outside of the game folder. It should have constructed filename in a different way.

[raziel-]

ohhh, goody

rhank you

[sev-]

It would be nice to get fixed before release

[sev-]

Oops, wrong ticket

[sev-]

Looking into this with Raziel. Mail sent.

[sev-]

In 754827c6:

COMMON: Avoid scanning out of game directory for AppleDouble files. Bug #15016

Also implement sanity check for paths containing drive, e.g. "Games:" on Amiga
or "D:" on Windows.

[sev-]

Fixed. It was an interesting find.

[mikrosk]

It seems that the issue hasn't been fully fixed: "_ _ MACOSX" is still searched on the level where scummvm(.exe) file is located.

For example, I have on C: the following tree:
a/

b/

scummvm

games/

rama-dos-win-ni-demo-en/

so when I do "cd a/b" and "./scummvm" and then choose "Add game" and go all the way up to "games/rama-dos-win-ni-demo-en" and click "Choose", there are multiple (up to 20!) requests of folder "a/b/_ _ MACOSX".

[raziel-]

@mikrosk

i think this is the intended way

the problem that was fixed was, that it created a search path looking like
_ _MACOSXmyhdd:a/b/
which produced errors because it was obviously non-existant.

looking for a subdir called _ _MACOSX in a given subdir is fine, i guess

[mikrosk]

Both #14843 (marked as duplicate of this one) and this ticket explicitly mention "Avoid scanning out of game directory" and this is exactly what is happening, scanning outside of game directory.

[sev-]

This is normal and expected. This is how our file search is working. You may mix files from different directories, e.g. extrapath and scummvm directory.