[Bug 41383] New: normal computation with vertex blending looks wrong

https://bugs.winehq.org/show_bug.cgi?id=41383

Bug ID: 41383
           Summary: normal computation with vertex blending looks wrong
           Product: Wine
           Version: unspecified
          Hardware: x86
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: directx-d3d
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

Hi,

In wined3d/glsl_shader.c, in shader_glsl_generate_ffp_vertex_shader, one can
find:

if (settings->normal)
{
    if (!settings->vertexblends)
    {
        shader_addline(buffer, "normal = ffp_normal_matrix *
ffp_attrib_normal;\n");
    }
    else
    {
        for (i = 0; i < settings->vertexblends + 1; ++i)
            shader_addline(buffer, "normal += ffp_attrib_blendweight[%u] *
(mat3(ffp_modelview_matrix[%u]) * ffp_attrib_normal);\n", i, i);
    }

if (settings->normalize)
        shader_addline(buffer, "normal = normalize(normal);\n");
}

There seems to be a mistake: in the non vertexblends case, ffp_normal_matrix is
set to the transpose of the inverse of the modelview matrix. That is in
accordance to the spec.
However in the vertexblends case, the modelview matrices are used non-inversed.

I guess they should be inversed and transposed (which explains why msdn says
half the number of blend matrices are available when normals are used: because
you need to store the inverses).

This non-msdn documentation
https://user.xmission.com/~legalize/book/download/06-Vertex%20Transformation...
(see page 14) also says the matrices should be inversed.

[Bug 41383] normal computation with vertex blending looks wrong

https://bugs.winehq.org/show_bug.cgi?id=41383

Józef Kucia  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #1 from Józef Kucia  ---
(In reply to Axel D from comment #0)
> There seems to be a mistake: in the non vertexblends case, ffp_normal_matrix
> is set to the transpose of the inverse of the modelview matrix. That is in
> accordance to the spec.
> However in the vertexblends case, the modelview matrices are used
> non-inversed.

That's what you would expect when implementing vertex blending. The inverse of
model view matrices was used in the the initial version of the patch [1].
However, some testing suggested that normals are simply transformed by model
view matrices [2]. It could be a good idea to add some tests for normals
transformation when vertex blending is enabled.

[1] - https://www.winehq.org/pipermail/wine-patches/2015-July/140489.html
[2] - https://www.winehq.org/pipermail/wine-devel/2015-July/108239.html

[Bug 41383] normal computation with vertex blending looks wrong

https://bugs.winehq.org/show_bug.cgi?id=41383

--- Comment #2 from Axel D  ---
It sounds indeed a good idea. Also the dx9 behaviour could differ from dx7/dx8.

[Bug 41383] normal computation with vertex blending looks wrong

https://bugs.winehq.org/show_bug.cgi?id=41383

Axel D  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |INVALID
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #3 from Axel D  ---
I have tested the dx9 samples SkinnedMesh and VertexBlend.
It seems that indeed the modelview matrices are used non-inversed.

In fact one can see that the vertex shaders generated for the "vertex shader"
case use the same matrices for the transformation of the normal and vertices.
This could have been ok if the matrices were orthogonal, but that's not the
case.

What is weird is that contradicts msdn. Besides the documentation for the
equivalent GL extension: GL_EXT_vertex_weighting says the transpose inverse are
weighted for the normal.

Perhaps one thing that would be interesting to check is the case with 0WEIGHTS
(which is supposed to be equivalent to no vertex blending, but perhaps because
of the normal calculation thing it is not.)

[Bug 41383] normal computation with vertex blending looks wrong

https://bugs.winehq.org/show_bug.cgi?id=41383

--- Comment #4 from Matteo Bruni  ---
(In reply to Axel D from comment #3)
> What is weird is that contradicts msdn.

Just for the records, notice that MSDN is often plain wrong.

[Bug 41383] normal computation with vertex blending looks wrong

https://bugs.winehq.org/show_bug.cgi?id=41383

Bruno Jesus  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #5 from Bruno Jesus  ---
Closing invalid bugs.