[Bug 52684] New: Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

Bug ID: 52684
           Summary: Command and Conquer Generals (and C&C Zero Hour) have
                    no ground textures
           Product: Wine
           Version: 7.3
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: directx-d3d
          Assignee: [e-mail]
          Reporter: [e-mail]
   Regression SHA1: 1632b8e7a419ebf256c4a356a58d505ef918f2fe
      Distribution: Slackware

Introduced in wine-7.3 with commit 1632b8e7a419ebf256c4a356a58d505ef918f2fe,
the C&C Generals game engine does not draw ground textures.  The ground (or
water) area is black.  Reverting to wine-7.2, or applying this commit as a
reverse-patch on wine-7.4 will fix the issue.  The commit was found by
bisecting the git tree.

This symptom appears in older bug reports but the fixes there do not appear to
work now.

Tested on two platforms, but both were Intel i915 graphics.  If necessary, I
can get access to an Nvidia platform.

To reproduce:
1. A free demo is available for download to run the game:
https://www.moddb.com/downloads/start/111369
(https://www.moddb.com/games/cc-generals/downloads/command-conquer-generals-d...).
2. Install in 32-bit prefix.
3. Run with `wine start generals.exe -nologo` and the game engine will be seen
running behind the main menu.  The ground will be blank if the bug is present.

Verified to affect:
https://appdb.winehq.org/objectManager.php?sClass=version&iId=4881
https://appdb.winehq.org/objectManager.php?sClass=version&iId=35407

Likely also:
https://appdb.winehq.org/objectManager.php?sClass=version&iId=1717

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

Zebediah Figura  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |regression
                 CC|                            |[e-mail]

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

Tijmen  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #1 from Tijmen  ---
I can confirm that this issue also exists for the NVIDIA platform using an MSI
GTX970.

Details:
Wine version wine-7.4 (package version wine 7.4-1)
Linux kernel 5.16.16-arch-1-1
NVIDIA driver version 510.54
NVML version 11.510.54
Xorg server version number 11.0
Xorg server vendor version 1.21.1.3

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #2 from Tijmen  ---
Created attachment 72071
  --> https://bugs.winehq.org/attachment.cgi?id=72071
Output of the program on stdout

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #3 from Tijmen  ---
Created attachment 72072
  --> https://bugs.winehq.org/attachment.cgi?id=72072
Screenshot illustrating the missing ground/water textures

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #4 from Zebediah Figura  ---
I can't seem to launch the program. It spits out a message box from wine's
msvcrt saying "Runtime error!" That's with a 32-bit prefix and running the
program using `wine start generals.exe -nologo` as mentioned.

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #5 from [e-mail] ---
I'm guessing you did NOT run the game when prompted by the installer, and
instead ran it afterward manually (and it failed to run).  I was able to
recreate this scenario a few times.

The problem seems to be, the game sometimes needs to find an Options.ini file
in the wine user Documents area.  I don't know if content is required, but the
minimal content the installer adds one first run is:

cat > ${WINEPREFIX}/drive_c/users/${USER}/Documents/Command\ and\ Conquer\
Generals\ Data/Option.ini <<EOF
IdealStaticGameLOD = High
StaticGameLOD = High
EOF
<Ctrl-D>

If this doesn't work, I can package a WINEPREFIX for you.

Note: the -nologo option just skips the intro movie.  It becomes annoying when
doing regression testing.

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #6 from [e-mail] ---
(In reply to bender647 from comment #5)
> cat > ${WINEPREFIX}/drive_c/users/${USER}/Documents/Command\ and\ Conquer\
> Generals\ Data/Option.ini <<EOF
> IdealStaticGameLOD = High
> StaticGameLOD = High
> EOF
>
> <Ctrl-D>

Sorry, typo above:  the file name needs to be Options.ini (not Option.ini).

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

Zebediah Figura  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
             Status|UNCONFIRMED                 |NEW

--- Comment #7 from Zebediah Figura  ---
(In reply to bender647 from comment #5)
> I'm guessing you did NOT run the game when prompted by the installer, and
> instead ran it afterward manually (and it failed to run).  I was able to
> recreate this scenario a few times.

I'm pretty sure I did run the game when prompted, and it crashed the same way.
Regardless, creating the Options.ini file as instructed did fix the crash, and
allowed me to reproduce the bug.

This one actually turned out into a whole mess of related problems. The basic
problem is that the game creates a SYSMEM and a DEFAULT texture, maps the
SYSMEM with D3DLOCK_NO_DIRTY_UPDATE, then does IDirect3DDevice8::CopyRects() to
copy it into the DEFAULT texture. There is one bug and two missed optimizations
here, any one of which would be sufficient to avoid this:

(1) Normally the sysmem texture would be left in WINED3D_LOCATION_SYSMEM after
the map, but because of the flag, we don't invalidate ~map_binding, hence
CLEARED is valid. Based on some testing I believe this code path is incorrect,
although I'm not sure it was possible to trigger an actual bug before the
offending commit. Unfortunately we rely on that lack of invalidation in some
other places, so those will also need to be fixed. In particular I believe we
should be respecting the dirty regions for managed textures in
wined3d_texture_load_location(), not when mapping. Even for managed textures
SYSMEM should still be valid; it's just not uploaded to the GPU.

(2) In order to blit from a CPU texture in SYSMEM to a GPU-only texture in
CLEARED, we end up loading the former into TEXTURE_RGB and doing a GPU-side
blit. That should be avoided. We don't want to load the GPU texture into a CPU
location, but we should be able to upload directly from the sysmem. Normally we
do, but in this case texture2d_blt() thinks that since CLEARED is CPU
accessible that this is a job for the CPU blitter.

(3) When loading a texture into TEXTURE_RGB whose valid locations are SYSMEM
and CLEARED, we end up clearing the texture again, which can be avoided (if
both of those locations really were valid, then SYSMEM is already cleared).
This may not be worth optimizing, though, since it'd take some really weird
usage to get here after fixing (1). [You'd have to do something like create a
staging texture, map it read-only without initializing it, and then copy from
it.]

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #8 from Henri Verbeet  ---
(In reply to Zebediah Figura from comment #7)
> (3) When loading a texture into TEXTURE_RGB whose valid locations are SYSMEM
> and CLEARED, we end up clearing the texture again, which can be avoided (if
> both of those locations really were valid, then SYSMEM is already cleared).
> This may not be worth optimizing, though, since it'd take some really weird
> usage to get here after fixing (1). [You'd have to do something like create
> a staging texture, map it read-only without initializing it, and then copy
> from it.]

Uploading from SYSMEM to TEXTURE_RGB in this case may not be quite optimal
either. It would be better than first clearing SYSMEM and then uploading from
there, but ideally we'd just clear on the GPU here. The GPU clear would almost
certainly be a "fast clear", which would be practically free.

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #9 from Zebediah Figura  ---
(In reply to Henri Verbeet from comment #8)
> (In reply to Zebediah Figura from comment #7)
> > (3) When loading a texture into TEXTURE_RGB whose valid locations are SYSMEM
> > and CLEARED, we end up clearing the texture again, which can be avoided (if
> > both of those locations really were valid, then SYSMEM is already cleared).
> > This may not be worth optimizing, though, since it'd take some really weird
> > usage to get here after fixing (1). [You'd have to do something like create
> > a staging texture, map it read-only without initializing it, and then copy
> > from it.]
>
> Uploading from SYSMEM to TEXTURE_RGB in this case may not be quite optimal
> either. It would be better than first clearing SYSMEM and then uploading
> from there, but ideally we'd just clear on the GPU here. The GPU clear would
> almost certainly be a "fast clear", which would be practically free.

Yeah, that's definitely the more salient optimization.

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #10 from Zebediah Figura  ---
(In reply to Zebediah Figura from comment #7)
> (1) Normally the sysmem texture would be left in WINED3D_LOCATION_SYSMEM
> after the map, but because of the flag, we don't invalidate ~map_binding,
> hence CLEARED is valid. Based on some testing I believe this code path is
> incorrect, although I'm not sure it was possible to trigger an actual bug
> before the offending commit. Unfortunately we rely on that lack of
> invalidation in some other places, so those will also need to be fixed. In
> particular I believe we should be respecting the dirty regions for managed
> textures in wined3d_texture_load_location(), not when mapping. Even for
> managed textures SYSMEM should still be valid; it's just not uploaded to the
> GPU.

One snag here is that the dirty regions are currently tracked on the client
thread. In order to fix this we'd need to move that to the CS thread.
Alternatively, we could move the entirety of MANAGED resource handling out of
the CS thread (which probably means explicitly keeping two different
wined3d_texture objects and blitting from one to the other as necessary), which
is arguably cleaner in some ways but does have its own complications.

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #11 from Henri Verbeet  ---
(In reply to Zebediah Figura from comment #10)
> (In reply to Zebediah Figura from comment #7)
> > (1) Normally the sysmem texture would be left in WINED3D_LOCATION_SYSMEM
> > after the map, but because of the flag, we don't invalidate ~map_binding,
> > hence CLEARED is valid. Based on some testing I believe this code path is
> > incorrect, although I'm not sure it was possible to trigger an actual bug
> > before the offending commit. Unfortunately we rely on that lack of
> > invalidation in some other places, so those will also need to be fixed. In
> > particular I believe we should be respecting the dirty regions for managed
> > textures in wined3d_texture_load_location(), not when mapping. Even for
> > managed textures SYSMEM should still be valid; it's just not uploaded to the
> > GPU.
>
> One snag here is that the dirty regions are currently tracked on the client
> thread. In order to fix this we'd need to move that to the CS thread.
> Alternatively, we could move the entirety of MANAGED resource handling out
> of the CS thread (which probably means explicitly keeping two different
> wined3d_texture objects and blitting from one to the other as necessary),
> which is arguably cleaner in some ways but does have its own complications.

I don't know the details of the tests you did, but should we perhaps simply be
marking the entire texture dirty upon creation?

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #12 from Zebediah Figura  ---
(In reply to Henri Verbeet from comment #11)
> I don't know the details of the tests you did, but should we perhaps simply
> be marking the entire texture dirty upon creation?

It'd help this bug, but I don't think it addresses the problem in general. I'm
not sure there are ways to trigger the bug in wined3d afterwards, but I don't
think it's correct not to invalidate ~map_binding. The comment in
add_dirty_rect_test() is relevant here:

* Side note, not tested in the test: Partial surface updates work, and two
separate
 * dirty rectangles are tracked individually. Tested on Nvidia Kepler, other
drivers
 * untested.

I was only able to test on WARP, but it appears to work the same way.

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

RobertX  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #13 from RobertX  ---
The problem with certain invisible textures have resurfaced since development
version 7.3 according to recollection.

Before, the problem did exist in the past, but was solved when the following
environment variables were inputted:

export MESA_GL_VERSION_OVERRIDE=4.5
export MESA_GLSL_VERSION_OVERRIDE=450
export WINEARCH=win32 wine

After 7.2, Wine Development does seem to ignore those lines of text and the
problem resurfaced.

Thank you.

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #14 from RobertX  ---
If you allow me to ask, can this bug pass into the next stable version of Wine?

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #15 from Zeb Figura  ---
(In reply to RobertX from comment #14)
> If you allow me to ask, can this bug pass into the next stable version of
> Wine?

The regression isn't in the 7.0.x stable branch. I've been trying to figure out
a solution for the development branch, but run into some difficulties as well
as time constraints. I'm sorry for the delay. I'm hoping to have it fixed by
the next release, at least (7.8)...

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #16 from RobertX  ---
OK. Thanks for the response.

Hope I didn't sound too demanded for something that is offered to us for free.

I am forgetting my manners. :(

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #17 from Zeb Figura  ---
No need to apologize; really a month is far longer of a turnaround time on this
bug than I intended...

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #18 from RobertX  ---
Is there anything we can do to assist?

I don't mind making a sacrifice if it will help out with the effort.

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

Zeb Figura  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
      Fixed by SHA1|                            |27e84c28972ab53e11c932cb67f
                   |                            |f5182894c1f27
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #19 from Zeb Figura  ---
Sorry, this is actually fixed upstream by
https://source.winehq.org/git/wine.git/commitdiff/27e84c28972ab53e11c932cb67ff5182894c1f27;
the fix should be released as part of 7.8.

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

Alexandre Julliard  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #20 from Alexandre Julliard  ---
Closing bugs fixed in 7.8.

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #21 from RobertX  ---
(In reply to Zeb Figura from comment #19)
> Sorry, this is actually fixed upstream by
> https://source.winehq.org/git/wine.git/commitdiff/
> 27e84c28972ab53e11c932cb67ff5182894c1f27; the fix should be released as
> part of 7.8.

Zeb Figura, I like you.

[Bug 52684] Command and Conquer Generals (and C&C Zero Hour) have no ground textures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52684

--- Comment #22 from [e-mail] ---
(In reply to Alexandre Julliard from comment #20)
> Closing bugs fixed in 7.8.

Confirmed fixed on 7.8, thank you.