[Bug 25528] New: HLSL shader not translated correctly (uniform variables)

http://bugs.winehq.org/show_bug.cgi?id=25528

Summary: HLSL shader not translated correctly (uniform
                    variables)
           Product: Wine
           Version: 1.3.5
          Platform: x86
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: directx-d3d
        AssignedTo: [e-mail]
        ReportedBy: [e-mail]

Created an attachment (id=32501)
 --> (http://bugs.winehq.org/attachment.cgi?id=32501)
Program run in a normal way

I was trying to run the game "Sexy Beach Zero" (hmmyes) and it runs great. But
when you load any of the 3D scenes, the body model isn't bound correctly to her
and is stationary at (0, 0, 0) in the scene.

I checked the log and found a lot of these:

err:d3d_shader:set_glsl_shader_program >>>>>>>>>>>>>>>>> GL_INVALID_OPERATION
(0x502) from Find glsl program uniform locations @ glsl_shader.c / 4435
err:d3d_shader:set_glsl_shader_program >>>>>>>>>>>>>>>>> GL_INVALID_OPERATION
(0x502) from glUseProgramObjectARB(programId) @ glsl_shader.c / 4449
err:d3d_shader:hardcode_local_constants >>>>>>>>>>>>>>>>> GL_INVALID_OPERATION
(0x502) from Hardcoding local constants @ glsl_shader.c / 3955
err:d3d_shader:shader_glsl_select >>>>>>>>>>>>>>>>> GL_INVALID_OPERATION
(0x502) from glUseProgramObjectARB @ glsl_shader.c / 4618
trace:d3d_constants:shader_glsl_load_constantsI Loading local constants 0: 2,
0, 0, 0
err:d3d_shader:shader_glsl_load_constantsI >>>>>>>>>>>>>>>>>
GL_INVALID_OPERATION (0x502) from glUniform4ivARB @ glsl_shader.c / 598
err:d3d_shader:shader_glsl_load_constants >>>>>>>>>>>>>>>>>
GL_INVALID_OPERATION (0x502) from glUniform4fvARB @ glsl_shader.c / 769

(After this it repeats the last 4 lines)

Perfectly willing to chip my teeth on this bug, I downloaded the latest Wine
git to check the source.

I ran the game again with the following command:

WINEDEBUG=trace+d3d_constants wine "Sexy Beach Zero English.exe" &> ~/sbz2.log

The log now returns this:

fixme:d3d_shader:print_glsl_info_log Error received from GLSL shader #10:
fixme:d3d_shader:print_glsl_info_log     Vertex info
fixme:d3d_shader:print_glsl_info_log     -----------
fixme:d3d_shader:print_glsl_info_log     0(35) : warning C1068: array index out
of bounds

Repeat the array index out of bounds warning plenty of times.

Going deeper, I ran the game like this:

WINEDEBUG=trace+d3d_shader wine "Sexy Beach Zero English.exe" &> ~/sbz.log

Part of the offending HLSL shader:

def *c822* = 1.000000, 3.000000, 0.000000, 0.000000
def c823 = 0.500000, 0.000000, 0.000000, 0.000000
def c824 = -1.000000, -2.000000, -3.000000, -4.000000
defi i0 = 5, 0, 0, 0

frc r0, v2
slt r1, -r0, r0
add r2, -r0, v2
slt r0, v2, -v2
mad r0, r0, r1, r2
mul r0, r0, *c822*.y

Emphasis marked with *'s.

uniform vec4 *VC[256]*;
// snip
void main() {
R0.xyzw = (fract(attrib2.xyzw));
R1.xyzw = (vec4(lessThan(-R0.xyzw, R0.xyzw)));
R2.xyzw = (-R0.xyzw + attrib2.xyzw);
R0.xyzw = (vec4(lessThan(attrib2.xyzw, -attrib2.xyzw)));
R0.xyzw = ((R0.xyzw * R1.xyzw) + R2.xyzw);
R0.xyzw = (R0.xyzw * *VC[822]*.yyyy);

This is where the "array index out of bounds" warning comes from. The shader
translator should have converted "c822" to a uniform, but instead put it in an
array.

I got hopelessly lost in the Wine source, I couldn't find the error myself,
unfortunately. :(

Hopefully this helps someone more knowledgeable with these things!

[Bug 25528] HLSL shader not translated correctly (uniform variables)

http://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #1 from Quinten Lansu  2010-12-15 14:06:12 CST ---
Created an attachment (id=32503)
 --> (http://bugs.winehq.org/attachment.cgi?id=32503)
Program run with debug info

[Bug 25528] HLSL shader not translated correctly (uniform variables)

http://bugs.winehq.org/show_bug.cgi?id=25528

Dan Kegel  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #2 from Dan Kegel  2010-12-15 15:15:43 CST ---
If you have an nvidia card, does 'winetricks glsl-disable' work around this?

[Bug 25528] HLSL shader not translated correctly (uniform variables)

http://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #3 from Quinten Lansu  2010-12-15 15:29:14 CST ---
Created an attachment (id=32505)
 --> (http://bugs.winehq.org/attachment.cgi?id=32505)
Program run with GLSL disabled

[Bug 25528] HLSL shader not translated correctly (uniform variables)

http://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #4 from Quinten Lansu  2010-12-15 15:29:29 CST ---
(In reply to comment #2)
> If you have an nvidia card, does 'winetricks glsl-disable' work around this?

I have an NVIDIA GeForce 9600 GT.

Unfortunately, it made things worse. The program now crashes on startup.

I've attached the log.

[Bug 25528] HLSL shader not translated correctly (uniform variables)

http://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #5 from Henri Verbeet  2010-12-15 15:47:13 CST ---
I'm not sure that's supposed to work (for hardware shaders anyway). Tests would
be helpful.

[Bug 25528] HLSL shader not translated correctly (uniform variables)

http://bugs.winehq.org/show_bug.cgi?id=25528

Skotlex  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #6 from Skotlex  2012-07-20 10:20:11 CDT ---
I've hit this bug (I believe) in a similar game (Artificial Academy) from the
same company, so I imagine it's the same issue. Is it all right to expand on
this bug-report rather than create a new one?

Basically, on the character maker, the moment a 3D model needs to be displayed,
wine will start throwing the index-out-of-bound exceptions in GLSL.

If I disable glsl (I have a Nvidia Quadro FX 770M card), at the moment of
displaying a 3D model, the error I get instead is:

err:d3d_shader:shader_arb_generate_vshader >>>>>>>>>>>>>>>>>
GL_INVALID_OPERATION (0x502) from glProgramStringARB() @ arb_program_shader.c /
4271
fixme:d3d_shader:shader_arb_generate_vshader HW VertexShader Error at position
578: "line 29, char 15: error: out of bounds array access\n"
(after which point wine launches an exception and the program needs be
terminated).

It's the same(?) issue, a PARAM C[256] is defined, and then C[789].x is
attempted to be accessed.

I want to understand whether this is a bug that can be solved in wine, or it's
just going to be tagged as a badly programmed application that is not going to
be catered to.

I'll attach the output with/without GLSL so you can decide yourself.

[Bug 25528] HLSL shader not translated correctly (uniform variables)

http://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #7 from Skotlex  2012-07-20 10:22:20 CDT ---
Created attachment 41067
  --> http://bugs.winehq.org/attachment.cgi?id=41067
Output of AAMake (glsl enabled)

[Bug 25528] HLSL shader not translated correctly (uniform variables)

http://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #8 from Skotlex  2012-07-20 10:22:48 CDT ---
Created attachment 41068
  --> http://bugs.winehq.org/attachment.cgi?id=41068
Output of AAMake (glsl disabled)

[Bug 25528] HLSL shader not translated correctly (uniform variables)

http://bugs.winehq.org/show_bug.cgi?id=25528

[e-mail] changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 25528] HLSL shader not translated correctly (uniform variables)

http://bugs.winehq.org/show_bug.cgi?id=25528

Sergey Lapin  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #9 from Sergey Lapin  2013-11-02 14:29:44 CDT ---
Same issue with wine-1.7.4

[Bug 25528] HLSL shader not translated correctly (uniform variables)

https://bugs.winehq.org/show_bug.cgi?id=25528

Dick Tracey  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #10 from Dick Tracey  ---
Is this being worked on at all?
1.7.35 continues to have the same problem.

[Bug 25528] HLSL shader not translated correctly (uniform variables)

https://bugs.winehq.org/show_bug.cgi?id=25528

Marisa Kirisame  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #11 from Marisa Kirisame  ---
It appears I'm getting the same issues in Artificial Academy 2. I mistakenly
thought it had something to do with software vertex blending, but nope, that
wasn't it.

With render mode 1 character models seem to be stuck in reference pose but can
sort of rotate around with the camera, with render mode 2 only faces are
glitched out.

I disabled GLSL and I get just a mess of vertices while the log spams me with:

err:d3d:wined3d_debug_callback 0x1aa9fc0: "GL_INVALID_OPERATION error
generated. State(s) are invalid: fragment program.".
err:d3d_draw:drawStridedFast >>>>>>>>>>>>>>>>> GL_INVALID_OPERATION (0x502)
from glDrawElementsBaseVertex @ ../../../wine/dlls/wined3d/drawprim.c / 66

[Bug 25528] HLSL shader not translated correctly (uniform variables)

https://bugs.winehq.org/show_bug.cgi?id=25528

[e-mail] changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #12 from [e-mail] ---
Same problem with 1.7.55 version in Artificial Academy 2.

[Bug 25528] HLSL shader not translated correctly (uniform variables)

https://bugs.winehq.org/show_bug.cgi?id=25528

Sergey Isakov  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #13 from Sergey Isakov  ---
Created attachment 53763
  --> https://bugs.winehq.org/attachment.cgi?id=53763
Sexy Beach 3 visual artefacts

To illustrate the problem.
Wine-1.9.4-118-g36095db

[Bug 25528] HLSL shader not translated correctly (uniform variables)

https://bugs.winehq.org/show_bug.cgi?id=25528

[e-mail] changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #14 from [e-mail] ---
There is the issue with shader constant limit, which natively not that strict
than Wine currently imposes (documented for software vertex processing mode,
but even for hardware vertex processing mode the actually allowed limit on many
cards seems to be higher than that advertised through caps). For these
particular games it can be workarounded by this patch:
https://bugs.winehq.org/attachment.cgi?id=52845. More details are here:
https://bugs.winehq.org/show_bug.cgi?id=39057

I am not sure if the main patch from that bug
(https://bugs.winehq.org/attachment.cgi?id=53581) is required for these games
(for indexed vertex blending), but all the games mentioned here are known to
work fine if to apply both of them).

I do not know if some core bug for shader constant limits already exist?

[Bug 25528] HLSL shader not translated correctly (uniform variables)

https://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #15 from Matteo Bruni  ---
(In reply to swswine from comment #14)
> There is the issue with shader constant limit, which natively not that
> strict than Wine currently imposes (documented for software vertex
> processing mode, but even for hardware vertex processing mode the actually
> allowed limit on many cards seems to be higher than that advertised through
> caps).

No, for hardware shaders the limit is correct; see test_unsupported_shaders()
in d3d9/tests/device.c.
> I do not know if some core bug for shader constant limits already exist?

This bug is probably okay to keep track of that.

[Bug 25528] HLSL shader not translated correctly (uniform variables)

https://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #16 from Sergey Isakov  ---
(In reply to swswine from comment #14)
> There is the issue with shader constant limit, which natively not that
> strict than Wine currently imposes (documented for software vertex
> processing mode, but even for hardware vertex processing mode the actually
> allowed limit on many cards seems to be higher than that advertised through
> caps). For these particular games it can be workarounded by this patch:
> https://bugs.winehq.org/attachment.cgi?id=52845. More details are here:
> https://bugs.winehq.org/show_bug.cgi?id=39057

No it is not needed.
> I am not sure if the main patch from that bug
> (https://bugs.winehq.org/attachment.cgi?id=53581) is required for these
> games (for indexed vertex blending), but all the games mentioned here are
> known to work fine if to apply both of them).

YES! This patch is the key.
I took wine-1.9.4-216-ga9b5bbf and apply your patch.
Fine! No one artefact found. Images good, geometry not broken.

This bug can be accounted as a dupe of 39057 because the game works without
issues.
Illustrate?

[Bug 25528] HLSL shader not translated correctly (uniform variables)

https://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #17 from Skotlex  ---
> This bug can be accounted as a dupe of 39057 because the game works without
> issues.
> Illustrate?

Not necessarily, they are somewhat related, but point at different kinds of
problems.

Bug 39057 is about indexed software blending.

This bug (25528) is about software shader limits.

Some may want to tackle both issues on the same bug, some may not.

[Bug 25528] HLSL shader not translated correctly (uniform variables)

https://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #18 from Sergey Isakov  ---
It is related.
I made a trace +d3d_shader and got million lines of traces. No one "array index
out of bound"!
Tested with "Sexy Beach 3" while the bug about "Sexy Beach Zero" so I am not
sure if it is the same.

[Bug 25528] HLSL shader not translated correctly (uniform variables)

https://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #19 from [e-mail] ---
(In reply to Matteo Bruni from comment #15)
> (In reply to swswine from comment #14)
> > There is the issue with shader constant limit, which natively not that
> > strict than Wine currently imposes (documented for software vertex
> > processing mode, but even for hardware vertex processing mode the actually
> > allowed limit on many cards seems to be higher than that advertised through
> > caps).
>
> No, for hardware shaders the limit is correct; see
> test_unsupported_shaders() in d3d9/tests/device.c.

Is it possible that the game creates shader when software mode is on, then
turns the software mode off, using created shaders and this works (even if the
shaders are actually executed in software maybe)? As I remember studying the
full trace logs of some of these the game does not have software mode on while
rendering. Anyway, I realize that this requires more testing & discovering the
exact scenario of what game is doing.

If anyone can send me a full log of the game (with
WINEDEBUG=+d3d,+d3d9,+d3d_shaders), better without any custom patches applied,
I can discover the sequence how it creates the device and switches software
mode in respect to shader creation and usage.

[Bug 25528] HLSL shader not translated correctly (uniform variables)

https://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #20 from Sergey Isakov  ---
(In reply to swswine from comment #19)
> If anyone can send me a full log of the game (with
> WINEDEBUG=+d3d,+d3d9,+d3d_shaders), better without any custom patches
> applied, I can discover the sequence how it creates the device and switches
> software mode in respect to shader creation and usage.

Hi man, 
did you consider it is impossible? I start the game with +d3d,+d3d9 and stop it
when the log exceed 4Gb. I truncated it. First 50000lines+last 50000lines. But
the game didn't show me first 3d scene.
Second log +d3d_shader only but with 3D scene. Unpacked size = 860Mb.

[Bug 25528] HLSL shader not translated correctly (uniform variables)

https://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #21 from Sergey Isakov  ---
Created attachment 53785
  --> https://bugs.winehq.org/attachment.cgi?id=53785
Sexy Beach 3 trace logs

[Bug 25528] HLSL shader not translated correctly (uniform variables)

https://bugs.winehq.org/show_bug.cgi?id=25528

--- Comment #22 from Sergey Isakov  ---
Yes, the bug 39057 is the same and the patch from those bug resolves the broken
picture.
Still present with wine-4.0 if not apply the patch.

[Bug 25528] HLSL shader not translated correctly (uniform variables)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=25528

joaopa  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #23 from joaopa  ---
Does the bug still occur with wine-6.5?