Hello,
i am trying to fix #9337 [1] for VietCong game (game is crashing during loading 3D objects) for which exists community util which fixes it. This util was made to fix issues for VC on newer Windows - fixing this bug in Wine was just side effect. Anyway, i have talked to creator of this util (Brchi from The Archi Family) about it. The game originally use software vertexes only, but he edited the game and switched to hardware vertexes (which increased FPS rapidly on Windows), but he was facing the same problem - the game was crashing. Fortunately, he found that the game uses different code branches when hardware vertex are used - developers probably never finished support for hardware vertexes and that is reason why it is crashing when HW vertexes are used. He fixed game code so it works with HW vertexes (even in Wine).
The problem is that game uses only SW vertexes and these unfinished branches are called only when HW vertexes are used, which is not a problem for Windows (which support SW vertexes). But for some reason Wine uses HW vertexes even if game wants to use SW vertexes. Thats why the game works in Wine only with this util applied.
I can see from report [2] that game tries to use SW vertexes, which are unsupported by Wine. Can I simply edit some code in Wine (some D3D flag) so the game will think that it use SW vertexes, so after checking some flags by game it will not jump into some unfinished branches for HW vertexes? I just want to test if it is really caused by this. From application view, what is difference between using SW and HW vertexes, do you think that using HW vertexes with code done for SW vertexes will work?
[1] https://bugs.winehq.org/show_bug.cgi?id=9337 [2] https://bugs.winehq.org/show_bug.cgi?id=9337#c35
Thanks, Jan
On 06/08/17 17:19, Jan Havran wrote:
Can I simply edit some code in Wine (some D3D flag) so the game will think that it use SW vertexes, so after checking some flags by game it will not jump into some unfinished branches for HW vertexes? I just want to test if it is really caused by this. From application view, what is difference between using SW and HW vertexes, do you think that using HW vertexes with code done for SW vertexes will work?
This behavior should already be in place. If the app requsts a software vertex processing device and asks us for the device description later we should report that this is a software vertex processing device. We even do some software-like things if the app wants SWVP, like ignore D3DLOCK_DISCARD on buffers.
The game may check some derived properties though and expect some capability that SWVP has, but HWVP does not. The most likely case is support for more than 256 vertex shader constants. Modern GPUs support much more than that in OpenGL, but not in d3d8/9 on Windows. Because of this we filter this in dlls/d3d8/device.c, d3dcaps_from_wined3dcaps(). Look for D3D8_MAX_VERTEX_SHADER_CONSTANTF at the end. It is likely that this restriction is not in place on Windows with SWVP. You can try to just disable this line.
Another possible culprit is fixed function vertex blending using matrix indices. You can try to set MaxVertexBlendMatrices and MaxVertexBlendMatrixIndex to 4 and 256 respectively (check on Windows if those values are accurate, I might misremember them). This functionality is unimplemented in Wine, so expect rendering problems if the game uses this. It should be possible to implement this feature in our fixed function replacement shader though.
The bug report doesn't have a crash dump. The similarity to windows HWVP might mislead you. Try to compare your crash to the one of windows HWVP. in particular, see if the crash address is the same.
On 6.8.2017 20:36, Stefan Dösinger wrote:
On 06/08/17 17:19, Jan Havran wrote:
Can I simply edit some code in Wine (some D3D flag) so the game will think that it use SW vertexes, so after checking some flags by game it will not jump into some unfinished branches for HW vertexes? I just want to test if it is really caused by this. From application view, what is difference between using SW and HW vertexes, do you think that using HW vertexes with code done for SW vertexes will work?
This behavior should already be in place. If the app requsts a software vertex processing device and asks us for the device description later we should report that this is a software vertex processing device. We even do some software-like things if the app wants SWVP, like ignore D3DLOCK_DISCARD on buffers.
The game may check some derived properties though and expect some capability that SWVP has, but HWVP does not. The most likely case is support for more than 256 vertex shader constants. Modern GPUs support much more than that in OpenGL, but not in d3d8/9 on Windows. Because of this we filter this in dlls/d3d8/device.c, d3dcaps_from_wined3dcaps(). Look for D3D8_MAX_VERTEX_SHADER_CONSTANTF at the end. It is likely that this restriction is not in place on Windows with SWVP. You can try to just disable this line.
Wine is big magic to me right now, so thank you for your tips, I will try to test that.
Another possible culprit is fixed function vertex blending using matrix indices. You can try to set MaxVertexBlendMatrices and MaxVertexBlendMatrixIndex to 4 and 256 respectively (check on Windows if those values are accurate, I might misremember them). This functionality is unimplemented in Wine, so expect rendering problems if the game uses this. It should be possible to implement this feature in our fixed function replacement shader though.
Ok, I will test it. Unfrotunately I didnt find piece of code where the game decide which vertex processing it will use yet - maybe I will install Windows so I will be able to compare execution of branches between Windows and Linux.
The bug report doesn't have a crash dump. The similarity to windows HWVP might mislead you. Try to compare your crash to the one of windows HWVP. in particular, see if the crash address is the same.
Well, I was talking about my issue with Brchi and we came to the conclusion that it was crashing in the same function for both Linux (Wine) and Windows (with enabled HWVP). Unfortunately, there is version of his util without HWVP enabled (which doesnt work in Wine as I said, because the game use branches for HWVP here) and after that version with HWVP enabled only, but with fixed game code (by memory patching) so there is no way how can I test it on Windows (because I dont have version of the util where it is crashing for Windows, since there is no version with HWVP enabled and without fix of this issue at the same time).
Anyway, this game has plenty of problems on newer Windows, but I never heard about this issue even on Windows 10.
I will try some tests you described and report back as soon as possible.
Thanks for your time.
Bingo!
I finally managed to install Windows 7 on my laptop and tried some debugging (I was comparing it to Wine). You was right, I have found that the game calls some D3D8 function where it gets D3DCAPS8 structure. In that structure it tests MaxVertexBlendMatrixIndex (offset 0xAC) and depend on that value it executes one or another branch. On Windows MaxVertexBlendMatrixIndex is set always to 255 as it says MS Dev Center for SWVP [1] (well, it is for D3D9, but i think it should be the same as D3D8). But the problem is that this value is set to 0 in Wine ([1] says that 0 means following: "device does not support indexed matrices"). As I have never heard about this issue on Windows, I guess that indexed matrices are always supported on Windows. I tested SWVP and HWVP on both Windows and Wine and here are the results:
Vietcong (Windows) with SW Vertex: 255 Vietcong (Windows) with HW Vertex: 126 Vietcong (Wine) with SW Vertex: 0 Vietcong (Wine) with HW Vertex: 126
So the game expects (was tested) 255 in SWVP but it gets 0. The minimum value of MaxVertexBlendMatrixIndex for which the application runs fine is 127, while anything below it (like 126 for HWVP or 0 for Wine SWVP/HWVP) will execute some broken branch (but for certain soldier figures only).
When I set this value to 255 in dlls/d3d8/device.c the game do not crash anymore, but instead of that there are some rendering problems (as you predicted, so you was right again). See attachment in [2].
How hard would be to fix it for D3D8 (and D3D9)?
[1] https://msdn.microsoft.com/en-us/library/windows/desktop/bb206316(v=vs.85).a... [2] https://bugs.winehq.org/show_bug.cgi?id=9337#c44
On 6.8.2017 22:25, Jan Havran wrote:
On 6.8.2017 20:36, Stefan Dösinger wrote:
On 06/08/17 17:19, Jan Havran wrote:
Can I simply edit some code in Wine (some D3D flag) so the game will think that it use SW vertexes, so after checking some flags by game it will not jump into some unfinished branches for HW vertexes? I just want to test if it is really caused by this. From application view, what is difference between using SW and HW vertexes, do you think that using HW vertexes with code done for SW vertexes will work?
This behavior should already be in place. If the app requsts a software vertex processing device and asks us for the device description later we should report that this is a software vertex processing device. We even do some software-like things if the app wants SWVP, like ignore D3DLOCK_DISCARD on buffers.
The game may check some derived properties though and expect some capability that SWVP has, but HWVP does not. The most likely case is support for more than 256 vertex shader constants. Modern GPUs support much more than that in OpenGL, but not in d3d8/9 on Windows. Because of this we filter this in dlls/d3d8/device.c, d3dcaps_from_wined3dcaps(). Look for D3D8_MAX_VERTEX_SHADER_CONSTANTF at the end. It is likely that this restriction is not in place on Windows with SWVP. You can try to just disable this line.
Wine is big magic to me right now, so thank you for your tips, I will try to test that.
Another possible culprit is fixed function vertex blending using matrix indices. You can try to set MaxVertexBlendMatrices and MaxVertexBlendMatrixIndex to 4 and 256 respectively (check on Windows if those values are accurate, I might misremember them). This functionality is unimplemented in Wine, so expect rendering problems if the game uses this. It should be possible to implement this feature in our fixed function replacement shader though.
Ok, I will test it. Unfrotunately I didnt find piece of code where the game decide which vertex processing it will use yet - maybe I will install Windows so I will be able to compare execution of branches between Windows and Linux.
The bug report doesn't have a crash dump. The similarity to windows HWVP might mislead you. Try to compare your crash to the one of windows HWVP. in particular, see if the crash address is the same.
Well, I was talking about my issue with Brchi and we came to the conclusion that it was crashing in the same function for both Linux (Wine) and Windows (with enabled HWVP). Unfortunately, there is version of his util without HWVP enabled (which doesnt work in Wine as I said, because the game use branches for HWVP here) and after that version with HWVP enabled only, but with fixed game code (by memory patching) so there is no way how can I test it on Windows (because I dont have version of the util where it is crashing for Windows, since there is no version with HWVP enabled and without fix of this issue at the same time).
Anyway, this game has plenty of problems on newer Windows, but I never heard about this issue even on Windows 10.
I will try some tests you described and report back as soon as possible.
Thanks for your time.
Small update, value of MaxVertexBlendMatrixIndex in HWVP on both Windows and Wine is not real value from GetDeviceCaps, but, as Brchi told me, this function was hooked in VCStarter (that Vietcong community util) to set this variable to 126. Anyway, SWVP value should be real.
On 15.8.2017 20:57, Jan Havran wrote:
Bingo!
I finally managed to install Windows 7 on my laptop and tried some debugging (I was comparing it to Wine). You was right, I have found that the game calls some D3D8 function where it gets D3DCAPS8 structure. In that structure it tests MaxVertexBlendMatrixIndex (offset 0xAC) and depend on that value it executes one or another branch. On Windows MaxVertexBlendMatrixIndex is set always to 255 as it says MS Dev Center for SWVP [1] (well, it is for D3D9, but i think it should be the same as D3D8). But the problem is that this value is set to 0 in Wine ([1] says that 0 means following: "device does not support indexed matrices"). As I have never heard about this issue on Windows, I guess that indexed matrices are always supported on Windows. I tested SWVP and HWVP on both Windows and Wine and here are the results:
Vietcong (Windows) with SW Vertex: 255 Vietcong (Windows) with HW Vertex: 126 Vietcong (Wine) with SW Vertex: 0 Vietcong (Wine) with HW Vertex: 126
So the game expects (was tested) 255 in SWVP but it gets 0. The minimum value of MaxVertexBlendMatrixIndex for which the application runs fine is 127, while anything below it (like 126 for HWVP or 0 for Wine SWVP/HWVP) will execute some broken branch (but for certain soldier figures only).
When I set this value to 255 in dlls/d3d8/device.c the game do not crash anymore, but instead of that there are some rendering problems (as you predicted, so you was right again). See attachment in [2].
How hard would be to fix it for D3D8 (and D3D9)?
[1] https://msdn.microsoft.com/en-us/library/windows/desktop/bb206316(v=vs.85).a... [2] https://bugs.winehq.org/show_bug.cgi?id=9337#c44
On 6.8.2017 22:25, Jan Havran wrote:
On 6.8.2017 20:36, Stefan Dösinger wrote:
On 06/08/17 17:19, Jan Havran wrote:
Can I simply edit some code in Wine (some D3D flag) so the game will think that it use SW vertexes, so after checking some flags by game it will not jump into some unfinished branches for HW vertexes? I just want to test if it is really caused by this. From application view, what is difference between using SW and HW vertexes, do you think that using HW vertexes with code done for SW vertexes will work?
This behavior should already be in place. If the app requsts a software vertex processing device and asks us for the device description later we should report that this is a software vertex processing device. We even do some software-like things if the app wants SWVP, like ignore D3DLOCK_DISCARD on buffers.
The game may check some derived properties though and expect some capability that SWVP has, but HWVP does not. The most likely case is support for more than 256 vertex shader constants. Modern GPUs support much more than that in OpenGL, but not in d3d8/9 on Windows. Because of this we filter this in dlls/d3d8/device.c, d3dcaps_from_wined3dcaps(). Look for D3D8_MAX_VERTEX_SHADER_CONSTANTF at the end. It is likely that this restriction is not in place on Windows with SWVP. You can try to just disable this line.
Wine is big magic to me right now, so thank you for your tips, I will try to test that.
Another possible culprit is fixed function vertex blending using matrix indices. You can try to set MaxVertexBlendMatrices and MaxVertexBlendMatrixIndex to 4 and 256 respectively (check on Windows if those values are accurate, I might misremember them). This functionality is unimplemented in Wine, so expect rendering problems if the game uses this. It should be possible to implement this feature in our fixed function replacement shader though.
Ok, I will test it. Unfrotunately I didnt find piece of code where the game decide which vertex processing it will use yet - maybe I will install Windows so I will be able to compare execution of branches between Windows and Linux.
The bug report doesn't have a crash dump. The similarity to windows HWVP might mislead you. Try to compare your crash to the one of windows HWVP. in particular, see if the crash address is the same.
Well, I was talking about my issue with Brchi and we came to the conclusion that it was crashing in the same function for both Linux (Wine) and Windows (with enabled HWVP). Unfortunately, there is version of his util without HWVP enabled (which doesnt work in Wine as I said, because the game use branches for HWVP here) and after that version with HWVP enabled only, but with fixed game code (by memory patching) so there is no way how can I test it on Windows (because I dont have version of the util where it is crashing for Windows, since there is no version with HWVP enabled and without fix of this issue at the same time).
Anyway, this game has plenty of problems on newer Windows, but I never heard about this issue even on Windows 10.
I will try some tests you described and report back as soon as possible.
Thanks for your time.
Hi,
Good work isolating the issue :-)
Am 2017-08-15 um 20:57 schrieb Jan Havran:
How hard would be to fix it for D3D8 (and D3D9)?
It's not an evening task, but certainly doable. The feature has example code in the dx8 and 9 sdk. Someone even started a patch and attached it to bug 39057. MSDN documents the feature here: https://msdn.microsoft.com/de-de/library/windows/desktop/bb174594(v=vs.85).a...
I had a quick look at the patch attached to bugzilla. Here is a non-exhaustive list of concernes I have:
*) The test should be extended to cover cases of conflicting pipeline configuration, e.g. D3DRS_INDEXEDVERTEXBLENDENABLE is enabled but D3DRS_VERTEXBLEND is not, or both states are enabled but no weight data is contained in the vertices. The test doesn't test vertex declarations and if it is indeed the case that only D3DDECLTYPE_UBYTE4 index values are valid. The existing D3DFVF_LASTBETA_D3DCOLOR definition suggests that at least D3DDECLTYPE_D3DCOLOR is valid too.
The answer to some of those issues may be "it depends on the driver", in which case a comment describing the outcome is fine and the test doesn't have to actively check the feature.
*) Porting the test to d3d8 and ddraw would be a good idea, especially since the game you have is a d3d8 game. It shouldn't be too difficult to adapt, and it can wait until the d3d9 version of the test is agreed on.
*) I am torn on its use of UBOs for the matrices. The 256 shader constants (256 vec4's aka 1024 uniforms) of d3d9 GPUs should be enough for 32 matrices (this leaves 128 constants for other parameters). Those GPUs don't have UBOs anyway. I think d3d10 GPUs (necessary for UBO support) advertise 1024 constants or more (4096 uniforms), which should give you the 128 matrices you need for your game.
If however there are games that need the 255 matrices that SWVP offers and there are still drivers that have less than 2048 old-style uniforms you need a UBO codepath. If in addition there is no game that is happy with <= 32 matrices you might as well drop the codepath for legacy uniforms.
*) For the UBO path glBufferSubData may be worth a try, or mapping the buffer with GL_MAP_INVALIDATE_BUFFER_BIT and always re-uploading everything. Doing a GL_WRITE_ONLY map like it currently does will stall the pipeline.
*) I don't think it is necessary to run the test for SW, HW and mixed VP, doing it once should be ok since the result appears to be the same in all cases.
Am 2017-08-15 um 21:55 schrieb Jan Havran:
Small update, value of MaxVertexBlendMatrixIndex in HWVP on both Windows and Wine is not real value from GetDeviceCaps, but, as Brchi told me, this function was hooked in VCStarter (that Vietcong community util) to set this variable to 126. Anyway, SWVP value should be real.
Why does VCStarter set the value to 126 when it's known to cause a crash? I guess it is between a Rock (crash) and a hard place (broken rendering because the real HWVP value is much smaller).
Fwiw, wined3d.dll and d3d9.dll can be compiled for Windows, so with indexed blending support in Wine VCStarter could ship wined3d (under the terms of the LGPLv2)
Hello again!
On 16.8.2017 11:38, Stefan Dösinger wrote:
Hi,
Good work isolating the issue :-)
Am 2017-08-15 um 20:57 schrieb Jan Havran:
How hard would be to fix it for D3D8 (and D3D9)?
It's not an evening task, but certainly doable. The feature has example code in the dx8 and 9 sdk. Someone even started a patch and attached it to bug 39057. MSDN documents the feature here: https://msdn.microsoft.com/de-de/library/windows/desktop/bb174594(v=vs.85).a...
I did not know about this patch. I tested it and it works!
I had a quick look at the patch attached to bugzilla. Here is a non-exhaustive list of concernes I have:
*) The test should be extended to cover cases of conflicting pipeline configuration, e.g. D3DRS_INDEXEDVERTEXBLENDENABLE is enabled but D3DRS_VERTEXBLEND is not, or both states are enabled but no weight data is contained in the vertices. The test doesn't test vertex declarations and if it is indeed the case that only D3DDECLTYPE_UBYTE4 index values are valid. The existing D3DFVF_LASTBETA_D3DCOLOR definition suggests that at least D3DDECLTYPE_D3DCOLOR is valid too.
The answer to some of those issues may be "it depends on the driver", in which case a comment describing the outcome is fine and the test doesn't have to actively check the feature.
*) Porting the test to d3d8 and ddraw would be a good idea, especially since the game you have is a d3d8 game. It shouldn't be too difficult to adapt, and it can wait until the d3d9 version of the test is agreed on.
The original game uses d3d8, yes. But there exist community util (made by Brchi), which converts D3D8 calls into D3D9, so it would be nice to have it in both d3d8 and d3d9, but it is not required.
*) I am torn on its use of UBOs for the matrices. The 256 shader constants (256 vec4's aka 1024 uniforms) of d3d9 GPUs should be enough for 32 matrices (this leaves 128 constants for other parameters). Those GPUs don't have UBOs anyway. I think d3d10 GPUs (necessary for UBO support) advertise 1024 constants or more (4096 uniforms), which should give you the 128 matrices you need for your game.
The game will not crash with 128 matrices, yes, but it is a question if it will be rendered correctly. Maybe dump question, I do not understand D3D much.
If however there are games that need the 255 matrices that SWVP offers and there are still drivers that have less than 2048 old-style uniforms you need a UBO codepath. If in addition there is no game that is happy with <= 32 matrices you might as well drop the codepath for legacy uniforms.
Hmm, sounds kinda difficult :)
*) For the UBO path glBufferSubData may be worth a try, or mapping the buffer with GL_MAP_INVALIDATE_BUFFER_BIT and always re-uploading everything. Doing a GL_WRITE_ONLY map like it currently does will stall the pipeline.
*) I don't think it is necessary to run the test for SW, HW and mixed VP, doing it once should be ok since the result appears to be the same in all cases.
Am 2017-08-15 um 21:55 schrieb Jan Havran:
Small update, value of MaxVertexBlendMatrixIndex in HWVP on both Windows and Wine is not real value from GetDeviceCaps, but, as Brchi told me, this function was hooked in VCStarter (that Vietcong community util) to set this variable to 126. Anyway, SWVP value should be real.
Why does VCStarter set the value to 126 when it's known to cause a crash? I guess it is between a Rock (crash) and a hard place (broken rendering because the real HWVP value is much smaller).
I am not sure, but I was told that there were some problems with default HWVP value.
Fwiw, wined3d.dll and d3d9.dll can be compiled for Windows, so with indexed blending support in Wine VCStarter could ship wined3d (under the terms of the LGPLv2)
Hmm, interesting, thx for info. I will let him know.
Anyway, it is nice to see that the sollution (even not in the git at the moment) was found exactly 10 years after reporting bug :)
Thank you very much for your help, I will keep watching updates of patch from #39057
-
Jan Havran -
Stefan Dösinger