[Bug 14968] New: DrawPrim.c needs to handle more than 4 vertex shader prims causes crash

http://bugs.winehq.org/show_bug.cgi?id=14968

Summary: DrawPrim.c needs to handle more than 4 vertex shader
                    prims causes crash
           Product: Wine
           Version: 1.1.3
          Platform: PC
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: directx-d3d
        AssignedTo: [e-mail]
        ReportedBy: [e-mail]
                CC: [e-mail]

Draw Prim needs to handle this in drawStridedSlow :

The vertex shader definition in direct x calls for 16 vertex shader types

currently several game applications call for vertex shader 7 initialization and 
wine fails with :

fixme:d3d_draw:drawStridedSlow Should not get here as coordsToUse is two bits
only (7)!

which is a valid mode :

which should map to the 4s gl call in opengl

here is the Direct X definition for mode 7

D3DDECLTYPE_SHORT4
    Four-component, signed short expanded to (value, value, value, value).

The link for the definition is :

http://msdn.microsoft.com/en-us/library/bb172533(VS.85).aspx

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash

http://bugs.winehq.org/show_bug.cgi?id=14968

--- Comment #1 from Vitaliy Margolen   2008-08-23 20:15:03 ---
Which games affected and how?

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash

http://bugs.winehq.org/show_bug.cgi?id=14968

--- Comment #2 from chris ahrendt   2008-08-23 20:23:19 ---
the original everquest, age of conan

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash

http://bugs.winehq.org/show_bug.cgi?id=14968

chris ahrendt  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         Difficulty|---                         |Minutes

--- Comment #3 from chris ahrendt   2008-08-23 21:00:50 ---
This is a very easy thing to fix

go into drawprim.c 
and into the function

drawStridedSlow

then go to the line with

/* The coords to supply depend completely on the fvf / vertex shader */

and add  the cases for the switch from 16 down. Currently they are just four

there needs to be 16 cases...

simple fix gets rid of the error.

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash

http://bugs.winehq.org/show_bug.cgi?id=14968

Vitaliy Margolen  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |INVALID

--- Comment #4 from Vitaliy Margolen   2008-08-23 21:21:21 ---
You missed the most important part - what _IS THE PROBLEM_?

Closing invalid.

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash

http://bugs.winehq.org/show_bug.cgi?id=14968

Vitaliy Margolen  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #5 from Vitaliy Margolen   2008-08-23 21:21:50 ---
Invalid. No problem specified.

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash

http://bugs.winehq.org/show_bug.cgi?id=14968

chris ahrendt  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|CLOSED                      |UNCONFIRMED
         Resolution|INVALID                     |

--- Comment #6 from chris ahrendt   2008-08-23 22:04:14 ---
The problem causes everquest to crash and causes the above error in the error
logs.

This is not invalid and has a easy fix to repair  the error  as I specified
earlier...

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash in everquest and does not allow rendering

http://bugs.winehq.org/show_bug.cgi?id=14968

chris ahrendt  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|DrawPrim.c needs to handle  |DrawPrim.c needs to handle
                   |more than 4 vertex shader   |more than 4 vertex shader
                   |prims causes crash          |prims causes crash in
                   |                            |everquest and does not allow
                   |                            |rendering

--- Comment #7 from chris ahrendt   2008-08-23 22:05:45 ---
The problem was ALREADY specified... if you look

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash in everquest and does not allow rendering

http://bugs.winehq.org/show_bug.cgi?id=14968

Stefan Dösinger  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #8 from Stefan Dösinger   2008-08-23 22:23:42 ---
Hmm. Does the application use a fixed function texture coordinate specified
with D3DECLUSAGE_FLOAT4?

It should be simple to support this. Essentially the "int coordsToUse =
sd->u.s.texCoords[coordIdx].dwType + 1" is wrong, this needs a better mapping:

decltype:                   coordsToUse:
D3DDECLTYPE_FLOAT1          1
D3DDECLTYPE_FLOAT2          2
D3DDECLTYPE_FLOAT3          3
D3DDECLTYPE_FLOAT4          4
D3DDECLTYPE_D3DCOLOR        4
D3DDECLTYPE_UBYTE4          4
D3DDECLTYPE_SHORT2          2
D3DDECLTYPE_SHORT4          4
D3DDECLTYPE_UBYTE4N         4
D3DDECLTYPE_SHORT2N         2
D3DDECLTYPE_SHORT4N         4
D3DDECLTYPE_USHORT2N        2
D3DDECLTYPE_USHORT4N        4
D3DDECLTYPE_UDEC3           ???
D3DDECLTYPE_DEC3N           ???
D3DDECLTYPE_FLOAT16_2       2
D3DDECLTYPE_FLOAT16_4       4

Also the function calls below shouldn't be selected based on a switch-case
statement, but rather on the declaration itself. q, r, t and s should be read
in a different way too. The system used for providing other parameters like
diffuse_funcs sounds sane here, although some extra care is needed to keep
working without GL_ARB_multitexture support.

Also I strongly recommend writing a test here. Generally, fixed function
attributes with non-standard data types are spooky on Windows, so the real
problem may be somewhere else.

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash in everquest and does not allow rendering

http://bugs.winehq.org/show_bug.cgi?id=14968

Dmitry Timoshkov  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|[e-mail]          |
         Difficulty|Minutes                     |---

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash in everquest and does not allow rendering

http://bugs.winehq.org/show_bug.cgi?id=14968

--- Comment #9 from Austin English   2008-08-24 13:28:38 ---
(In reply to comment #3)
> This is a very easy thing to fix
>
> go into drawprim.c 
> and into the function
>
> drawStridedSlow
>
> then go to the line with
>
> /* The coords to supply depend completely on the fvf / vertex shader */
>
> and add  the cases for the switch from 16 down. Currently they are just four
>
> there needs to be 16 cases...
>
> simple fix gets rid of the error.

If it's a simple fix, create a patch and submit it to [e-mail]

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash in everquest and does not allow rendering

http://bugs.winehq.org/show_bug.cgi?id=14968

--- Comment #10 from chris ahrendt   2008-08-24 14:47:27 ---
Well its simple in that you can get the error to go away by doing the suggested
below. However you run into stephans question on implementing it correctly.

Chris

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash in everquest and does not allow rendering

http://bugs.winehq.org/show_bug.cgi?id=14968

--- Comment #11 from chris ahrendt   2008-09-04 11:34:20 ---
> Hmm. Does the application use a fixed function texture coordinate specified
> with D3DECLUSAGE_FLOAT4?

I don't know I can look in my logs and see if I see it...  
but don't remember off the top of my head at the moment.
> It should be simple to support this. Essentially the "int coordsToUse =
> sd->u.s.texCoords[coordIdx].dwType + 1" is wrong, this needs a better mapping:
> decltype:                   coordsToUse:
> D3DDECLTYPE_FLOAT1          1
> D3DDECLTYPE_FLOAT2          2
> D3DDECLTYPE_FLOAT3          3
> D3DDECLTYPE_FLOAT4          4
> D3DDECLTYPE_D3DCOLOR        4
> D3DDECLTYPE_UBYTE4          4
> D3DDECLTYPE_SHORT2          2
> D3DDECLTYPE_SHORT4          4
> D3DDECLTYPE_UBYTE4N         4
> D3DDECLTYPE_SHORT2N         2
> D3DDECLTYPE_SHORT4N         4
> D3DDECLTYPE_USHORT2N        2
> D3DDECLTYPE_USHORT4N        4
> D3DDECLTYPE_UDEC3           3 - 3 unsigned decimals 
> D3DDECLTYPE_DEC3N           3 - 3 normalized decimals  
> D3DDECLTYPE_FLOAT16_2       2
> D3DDECLTYPE_FLOAT16_4       4

The app. should pass these in to us already in the correct format
> Also the function calls below shouldn't be selected based on a switch-case
> statement, but rather on the declaration itself. q, r, t and s should be read
> in a different way too. The system used for providing other parameters like
> diffuse_funcs sounds sane here, although some extra care is needed to keep
> working without GL_ARB_multitexture support.

ok explain... I am lost here..  like I said earlier I divided it into the gl
calls 
(float, int, short) and the screen went a little bonkers...
so any suggestions here would be great =)
> Also I strongly recommend writing a test here. Generally, fixed function
> attributes with non-standard data types are spooky on Windows, so the real
> problem may be somewhere else.

Is there currently a test case for this written?  I think on windows it just
ignores the parms to be honest..

Chris

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash in everquest and does not allow rendering

http://bugs.winehq.org/show_bug.cgi?id=14968

--- Comment #12 from Austin English   2009-03-10 15:03:09 ---
Is this still an issue in current (1.1.16 or newer) wine?

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash in everquest and does not allow rendering

http://bugs.winehq.org/show_bug.cgi?id=14968

chris ahrendt  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |FIXED

--- Comment #13 from chris ahrendt   2009-03-11 15:43:43 ---
Looks like it is fixed in the latest GIT

[Bug 14968] DrawPrim.c needs to handle more than 4 vertex shader prims causes crash in everquest and does not allow rendering

http://bugs.winehq.org/show_bug.cgi?id=14968

Alexandre Julliard  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #14 from Alexandre Julliard   2009-03-13 11:15:37 ---
Closing bugs fixed in 1.1.17.