[Bug 42732] New: GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

Bug ID: 42732
           Summary: GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV
                    Heavensward Benchmark
           Product: Wine
           Version: 2.4
          Hardware: x86
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: -unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

Get the benchmark here: http://eu.finalfantasyxiv.com/benchmark/en

Requires d3dx9 and xact_jun2010, then directx9 installed with winetricks to
run.
d3dx9 and xact_jun2010 stops it complaining about DirectX not being installed,
while directx9 stops the benchmark crashing as soon as it starts.

First of all, I do not get this bug if I use my AMD RX 480 - it only happens on
my Intel HD 530.

Upon running the benchmark with Standard (Desktop) and windowed 1080p settings
(happens on every other setting combination I've tried too, though), the first
scene plays out fine (though with some occlusion culling issues like
flickering, if it's enabled), but while loading the second scene, it tried to
write to NULL with the following output on the console (that differs to a
successful run under my RX 480):

err:d3d:wined3d_debug_callback 0x10311a28: "GL_OUT_OF_MEMORY in
glCompressedTexSubImage2D".
(...repeated for many lines...)
err:d3d:wined3d_debug_callback 0x10311a28: "GL_OUT_OF_MEMORY in
glCompressedTexSubImage2D".
err:d3d:wined3d_debug_callback 0x10311a28: "GL_OUT_OF_MEMORY in glTexSubImage".
wine: Unhandled page fault on write access to 0x00000000 at address 0xf7457bec
(thread 0126), starting debugger...

So I thought this meant I didn't have enough VRAM, so I increased that in the
BIOS to 512 MB of dedicated VRAM and no limit to shared VRAM. I have 32 GB of
system memory.

I also changed HKCU\Software\Wine\Direct3D\VideoMemorySize to various different
values (between 128 and 4096) but still got the same error.

Also happens on both 32-bit and 64-bit wine prefixes.

[Bug 42732] GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

--- Comment #1 from Matteo Bruni  ---
(In reply to nospam from comment #0)
> err:d3d:wined3d_debug_callback 0x10311a28: "GL_OUT_OF_MEMORY in
> glCompressedTexSubImage2D".
> (...repeated for many lines...)

It probably means that the game process is exhausting its addressing space. I
guess there isn't a 64-bit version of the game?

You could try to enable the LARGE_ADDRESS_AWARE flag on the game executable
with one of the tools freely available, if it's not already set. Given that you
need to use native d3dx9 that might also crash though.

[Bug 42732] GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

--- Comment #2 from [e-mail] ---
I set the flag, and now the benchmark works with no problems.
(For the future reader, the benchmark executable is game/ffxiv.exe, not
ffxiv-heavensward-bench.exe)

I suppose now the question is, why is this needed under the Intel GPU and not
my AMD GPU? And of course, running the benchmark under Windows with the iGPU
doesn't cause a crash, so why does it happen under wine?

[Bug 42732] GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

[e-mail] changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #3 from [e-mail] ---
(In reply to nospam from comment #2)
> I set the flag, and now the benchmark works with no problems.
> (For the future reader, the benchmark executable is game/ffxiv.exe, not
> ffxiv-heavensward-bench.exe)
>
> I suppose now the question is, why is this needed under the Intel GPU and
> not my AMD GPU? And of course, running the benchmark under Windows with the
> iGPU doesn't cause a crash, so why does it happen under wine?

iGpu is using ram memory, one possibility is that the amount is set low
motherboard bios. It has some limit, it can't be really high. Other possibility
is that external gpu has its own memory on the gpu card and it's mapped
differently than iGpu's memory.

[Bug 42732] GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

--- Comment #4 from [e-mail] ---
Created attachment 57857
  --> https://bugs.winehq.org/attachment.cgi?id=57857
/proc/.../maps on the scene2 loading screen

[Bug 42732] GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

--- Comment #5 from [e-mail] ---
Created attachment 57858
  --> https://bugs.winehq.org/attachment.cgi?id=57858
/proc/.../maps while scene 2 is running

[Bug 42732] GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

[e-mail] changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #57857|0                           |1
        is obsolete|                            |

--- Comment #6 from [e-mail] ---
Created attachment 57859
  --> https://bugs.winehq.org/attachment.cgi?id=57859
/proc/.../maps on the scene 2 loading screen on the AMD GPU

[Bug 42732] GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

[e-mail] changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #57858|0                           |1
        is obsolete|                            |

--- Comment #7 from [e-mail] ---
Created attachment 57860
  --> https://bugs.winehq.org/attachment.cgi?id=57860
/proc/.../maps while scene 2 is running on the AMD GPU

[Bug 42732] GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

--- Comment #8 from [e-mail] ---
Created attachment 57861
  --> https://bugs.winehq.org/attachment.cgi?id=57861
/proc/.../maps on the scene 2 loading screen on the Intel GPU (unpatched)

[Bug 42732] GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

--- Comment #9 from [e-mail] ---
Created attachment 57862
  --> https://bugs.winehq.org/attachment.cgi?id=57862
/proc/.../maps on the scene 2 loading screen crash on the Intel GPU (unpatched)

[Bug 42732] GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

--- Comment #10 from [e-mail] ---
Created attachment 57863
  --> https://bugs.winehq.org/attachment.cgi?id=57863
/proc/.../maps on the scene 2 loading screen on the Intel GPU (patched)

[Bug 42732] GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

--- Comment #11 from [e-mail] ---
Created attachment 57864
  --> https://bugs.winehq.org/attachment.cgi?id=57864
/proc/.../maps while scene 2 is running on the Intel GPU (patched)

[Bug 42732] GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

--- Comment #12 from [e-mail] ---
I've attached the /proc/.../maps of the benchmark in various circumstances (and
with ASLR turned off to make things a bit more comparable).

The most major change between the loading screen and the scene itself are lots
of added "/dev/dri/renderDxxx" and "/drm mm object (deleted)" maps.

Using the following command line to count the number of bytes mapped in by
these maps:
grep -E '/(dri|drm)' | cut -d' ' -f1 | awk -F- '{print strtonum("0x" $2) -
strtonum("0x" $1)}' | tr '\n' + | sed -E 's/+$/\n/' | bc

We get the following numbers (loading, running/crashed, difference):
dGPU: 62, 77, 15 MiB
iGPU (unpatched): 199, 350, 151 MiB
iGPU (patched): 199, 529, 340 MiB

So indeed the dGPU seems to use less virtual memory than the iGPU, and the
crash is caused by failing to allocate the final 340 - 151 = 189 MiB of memory.

What strikes me as odd is in the unpatched case, all the dri/drm maps are
allocated in the lower 2 GiB of memory (in contrast to the patched case, where
most of it is in high memory). Isn't the native linux graphics driver
allocating this memory? Why would it be affected by the address space
limitations that wine imposes on Windows binaries?

[Bug 42732] GL_OUT_OF_MEMORY followed by NULL dereference in FFXIV Heavensward Benchmark

https://bugs.winehq.org/show_bug.cgi?id=42732

--- Comment #13 from Matteo Bruni  ---
(In reply to nospam from comment #12)
> What strikes me as odd is in the unpatched case, all the dri/drm maps are
> allocated in the lower 2 GiB of memory (in contrast to the patched case,
> where most of it is in high memory). Isn't the native linux graphics driver
> allocating this memory? Why would it be affected by the address space
> limitations that wine imposes on Windows binaries?

That's because Wine normally reserves most of the high 2GBs of the addressing
space. You can see it from your "unpatched" attachments:

80000000-f7c10000 ---p 00000000 00:00 0

AFAIUI the idea is to try to enforce all memory allocations to go in the lower
half of the addressing space so that the Windows application doesn't ever get
pointers to high addresses (which it generally doesn't expect and might cause
it to break more or less subtly).

If the LARGE_ADDRESS_AWARE flag is set, Wine drops the high memory map during
process initialization and makes the addressing space available for both native
and Win memory allocations.