SDL2/WIN10: Segfault after locking/unlocking the computer

[rootfather]

Note: This issue was first discovered by waltervn.

When building the latest ScummVM codebase with SDL2 and the msys2/mingw-w64 toolchain, ScummVM crashes with a segmentation fault after locking/unlocking the computer either due to inactivity or manually by pressing Win+L.

After unlocking, ScummVM crashes instantly in fullscreen mode. In windowed mode, ScummVM crashes as soon as the cursor is moved inside the ScummVM window. The crash occures when displaying the launcher as well as while an game engine is running.

My setup:
ScummVM revision 03cf470
SDL 2.0.4
GCC 6.1.0 from the msys2 toolchain
Windows 10, Insider Build 14905 [other builds might be affected too]
AMD integrated graphics

The crash occures with the SDL2.dll binary shipped with msys2 as well as the one inside the scummvm_libs2015 package and even the official one from libsdl.org (all .dlls are version 2.0.4).

Thankfully, the SDL2.dll from the ScummVM Libs package has debug symbols in it, so here's the backtrace right after unlocking the computer (and crashing ScummVM).

Starting program: C:\users\user\git\scummvm\scummvm.exe
[New Thread 22604.0x4564]
[New Thread 22604.0x4f74]
[New Thread 22604.0x55dc]
[New Thread 22604.0x42cc]
[New Thread 22604.0x1ca0]
[New Thread 22604.0x5b78]
[New Thread 22604.0x5fd0]
[New Thread 22604.0x5b34]
[New Thread 22604.0x5cc4]
[New Thread 22604.0x5d38]
[New Thread 22604.0x36c0]
[New Thread 22604.0x5cf4]
[New Thread 22604.0x5cac]
[New Thread 22604.0x441c]
[New Thread 22604.0x46e8]

Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ffac14f9bfd in SDL_UnlockSurface () from C:\msys64\mingw64\bin\SDL2.dll
(gdb) backtrace
#0  0x00007ffac14f9bfd in SDL_UnlockSurface ()
   from C:\msys64\mingw64\bin\SDL2.dll
#1  0x00007ffac14f5a31 in SDL_UnlockSurface ()
   from C:\msys64\mingw64\bin\SDL2.dll
#2  0x00007ffac14f94d7 in SDL_UnlockSurface ()
   from C:\msys64\mingw64\bin\SDL2.dll
#3  0x00007ffac14f89e8 in SDL_UnlockSurface ()
   from C:\msys64\mingw64\bin\SDL2.dll
#4  0x00007ffac14f1370 in SDL_UnlockSurface ()
   from C:\msys64\mingw64\bin\SDL2.dll
#5  0x00007ffac14b7184 in SDL_UnlockSurface ()
   from C:\msys64\mingw64\bin\SDL2.dll
#6  0x000000000046704d in SurfaceSdlGraphicsManager::SDL_UpdateRects (
    this=0x37ed070, screen=0x3778430, numrects=1, rects=0x37ed1dc)
    at backends/graphics/surfacesdl/surfacesdl-graphics.cpp:2491
#7  0x00000000004632aa in SurfaceSdlGraphicsManager::internUpdateScreen (
    this=0x37ed070)
    at backends/graphics/surfacesdl/surfacesdl-graphics.cpp:1261
#8  0x0000000000462572 in SurfaceSdlGraphicsManager::updateScreen (
    this=0x37ed070)
    at backends/graphics/surfacesdl/surfacesdl-graphics.cpp:1018
#9  0x0000000000458f14 in ModularBackend::updateScreen (this=0x37d5050)
    at backends/modular-backend.cpp:149
#10 0x0000000000414b89 in GUI::GuiManager::runLoop (this=0x3832920)
    at gui/gui-manager.cpp:304
#11 0x000000000040ffc5 in GUI::Dialog::runModal (this=0x1879d70)
    at gui/dialog.cpp:80
#12 0x0000000000404ad0 in launcherDialog () at base/main.cpp:96
#13 0x00000000004066b0 in scummvm_main (argc=1, argv=0x37c2b80)
    at base/main.cpp:481
#14 0x0000000000403a1d in SDL_main (argc=1, argv=0x37c2b80)
    at backends/platform/sdl/win32/win32-main.cpp:72
#15 0x0000000000403968 in WinMain ()
    at backends/platform/sdl/win32/win32-main.cpp:55
#16 0x00000000005d5772 in main (flags=<optimized out>,
    cmdline=<optimized out>, inst=<optimized out>)
    at C:/repo/mingw-w64-crt-git/src/mingw-w64/mingw-w64-crt/crt/crt0_c.c:18
#17 0x00000000004013f8 in __tmainCRTStartup ()
    at C:/repo/mingw-w64-crt-git/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:334
#18 0x00000000004014eb in WinMainCRTStartup ()
    at C:/repo/mingw-w64-crt-git/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:184

Unfortunately, I was unable to compile the latest SDL2 code from the HG repository due to compilation errors while compiling some DirectX components.

On the same machine, I don't have any issues with other SDL2 applications like MAME or RetroArch (though RetroArch is based on SDL 2.0.3)

[rootfather]

I found out that this crash only happens when using the SDL rendering methods.

There are no issues after locking/unlocking the computer when using OpenGL rendering.

[rootfather]

SDL 2.0.3

[rootfather]

The issue seems to be related to SDL2 2.0.4. When using the .dll for SDL2 2.0.3 (attached), the crash doesn't happen.

@waltervn, can you please test with SDL2 2.0.3?

Unfortunately, I'm unable to build the latest SDL HG revision, so I cannot check against the latest SDL2 codebase.

[rootfather]

SDL2 2.0.4 HG

[rootfather]

I managed to compile the latest SDL2 codebase and I attached both the 64- and 32 Bit binaries.

First, this is not a bug in ScummVM, but in SDL2 that got recently fixed. ​https://bugzilla.libsdl.org/show_bug.cgi?id=3147.

There are two reports in the SDL forums that show similar issues, see ​https://forums.libsdl.org/viewtopic.php?p=51329#51329 and ​https://forums.libsdl.org/viewtopic.php?t=11913.

The bug got fixed with SDL2 revision 17e0ded12e6f. ​https://hg.libsdl.org/SDL/rev/17e0ded12e6f

Using the HG DLLs I attached fixed the problem for me. However, there is a very small issue: After unlocking in windowed mode, the ScummVM window is black until you move the mouse cursor. I *assume* that this "black state" was what caused the crash in 2.0.4, because it seems that the fix in 17e0ded12e6f is more of a workaround.

I read something about "unsupported pixel formats" being the cause for the crash, but due to my limited knowledge, I cannot confirm if this applies for ScummVM or if there is something we can fix.

Conclusion: Not a ScummVM bug. Bug in SDL2 2.0.4. Fixed in later revisions.