3DS: make use of linear heap to make resource-intensive games playable

[BallM4788]

The 3DS has two types of heaps: a regular heap and a linear heap. ​Here is a visualization. Regular heap is used via the typical methods (new, malloc, calloc, realloc, free, etc.), while linear heap must be accessed via ​these ​functions provided by ctrulib.

On the O3DS (Original 3DS), the main program memory, the regular heap, and the linear heap combined can be no greater than 64MB by default; this can be increased up to 96MB when building ScummVM as a .cia file by ​editing the .rsf file. Similarly, on the N3DS (New 3DS), the combined memory cannot exceed 124MB by default, but can be increased to 178 by ​editing the .rsf file. However, per internal 3DS documentation from the Nintendo gigaleaks (find it yourself), the regular heap is capped to 96MB, even on the N3DS.

Currently, the 3DS port of ScummVM uses the default regular and linear heap sizes ​as defined in ctrulib (<=24MB and <=32MB, respectively). Not only does it mean that there is tons of available memory currently being unused (up to 152+MB, not accounting for stack memory and screen buffers in the linear heap), but it also means the TinyGL-powered games all segfault during initial loading, if not immediately. As far as I know, the only 3D game that can currently be run on the 3DS port is Blade Runner, and that's only because it uses voxel-based character models.

I have experimented with maxxing out the regular heap to its 96MB cap, and in both Grim Fandango and The Longest Journey, the intro cutscenes play in their entirety and the starting areas load up successfully (most of Manny's office in GF, and the Arcadia cliffside in TLJ). Attempting to progress beyond this (accessing Manny's inventory, moving right to the screen with his computer, or leaving his office in GF; walking right to the White Dragon's nest or further in TLJ) causes a segfault.

If the 3DS's linear heap could be used to hold memory-intensive data, such as textures, streaming data, etc., it might very well be possible to get all the 3D games to run in the 3DS port. Except for Penumbra Overture, of course. That would be a fool's errand.