Broken character models in dx8/9 games in wined3d
I've spent a couple of days researching the issue of broken/upside-down character/object models in Wine in almost all newer games when you have vertex shaders enabled (Civ4, Half Life 2, Oblivion, Max Payne 2, etc.). I think I've boiled it down to a single case: When "device->renderUpsideDown" is set in the case where vertex shaders are enabled. That flag gets set in device.c:7395 when the current renderTarget is not on the current swapchain. The comments in the source say that the upside-downedness is produced by glCopyTexImage, so it sets a flag to flip everything over. In the case w/o shaders, there is code in drawprim.c which loads the WORLDVIEW and PROJECTION matrices and then multiplies those matrices by one which inverts the y coordinates when that flag is set. That seems to work in the case without vertex shaders, but when shaders are enabled, they bypass the WORLD, VIEW, and PROJECTION matrices entirely. The shader case was written when only software shaders worked, but that is no longer true. It loads identity matrices and performs the y flip, but that code is entirely irrelevant since the vertex shader doesn't reference those matrices; it only uses constants that are passed by the app, which we can't perform any type of fixup on since we don't know which constants will be used for which calculation. So, I think what we need to do is prevent ourselves from having to do any flipping whatsoever. That's the part that I'm not sure how to do and is the reason for this email. Can we load the textures in system memory first, perform a software reversing process, then load that up with glCopyTexImage instead? Will we need to do that type of fixup every time the app locks/unlocks/changes part of the texture? Or, is there a better way? I think I've figured out the problem, it's just the next step of fixing it that I'm unsure of. :-)
From a few hackish tests, I've been able to fix some of the issues where the only thing wrong was that the images were upside-down with this hack (for GLSL shader mode only):
diff --git a/dlls/wined3d/vertexshader.c b/dlls/wined3d/vertexshader.c index 84f90f5..d0325d9 100644 --- a/dlls/wined3d/vertexshader.c +++ b/dlls/wined3d/vertexshader.c @@ -719,6 +719,7 @@ #endif shader_addline(&buffer, "gl_FogFragCoord = clamp(gl_FogFragCoord, 0.0, 1.0);\n"); } + shader_addline(&buffer, "gl_Position = gl_Position * gl_ModelViewProjectionMatrix;\n"); shader_addline(&buffer, "}\n\0"); TRACE("Compiling shader object %u\n", shader_obj); This works because the gl_ModelViewProjectionMatrix is just a matrix which reverses the y position when This->renderUpsideDown == TRUE, otherwise it's the identity matrix. Here are a few comparison screenshots: Oblvion: --------- Before: http://www.cmhousing.net/wine/oblivion2.png After: http://www.cmhousing.net/wine/oblivion_sortacorrect3.png Civ4: ----- Before: http://www.cmhousing.net/wine/civ4_before.png After: http://www.cmhousing.net/wine/civ4_leaderhead.png Half Life 2: ------------- After: http://www.cmhousing.net/wine/hl2_rightsideup.png (the bottom got screwed up, but normally the guy on the picture is upside-down) Many of the character models and other objects are still broken even with that hack, but that's because we should be flipping more than just the final position - we should be flipping a whole row of constants that the app is using before it starts its calculations. Instead of trying to do that, we should fix the root of the problem IMHO. Anyway, any ideas are welcome. :-) Jason
On 7/24/06, Jason Green <jave27(a)gmail.com> wrote:
I've spent a couple of days researching the issue of broken/upside-down character/object models in Wine in almost all newer games when you have vertex shaders enabled (Civ4, Half Life 2, Oblivion, Max Payne 2, etc.).
Yes, I remember some talk on IRC, and wanted to comment, but will do now.
I think I've boiled it down to a single case: When "device->renderUpsideDown" is set in the case where vertex shaders are enabled. That flag gets set in device.c:7395 when the current renderTarget is not on the current swapchain. The comments in the source say that the upside-downedness is produced by glCopyTexImage, so it sets a flag to flip everything over.
In the case w/o shaders, there is code in drawprim.c which loads the WORLDVIEW and PROJECTION matrices and then multiplies those matrices by one which inverts the y coordinates when that flag is set. That seems to work in the case without vertex shaders, but when shaders are enabled, they bypass the WORLD, VIEW, and PROJECTION matrices entirely. The shader case was written when only software shaders worked, but that is no longer true. It loads identity matrices and performs the y flip, but that code is entirely irrelevant since the vertex shader doesn't reference those matrices; it only uses constants that are passed by the app, which we can't perform any type of fixup on since we don't know which constants will be used for which calculation.
Yeah, I've noticed all this with Star Wars Battlefront. See bug 5247. Without vertex shaders, I have a very simple hack to fix the one issue of an upside down skybox. What I did should be quite obvious to you. With vertex shaders (note: I make brief mention in bug comments about how to get them working with dri, you might remember this from IRC too) having them enabled, the problem gets a little worse as certain parts of the skybox are correctly up and others upside down. If you apply the hack, then *everything* got flipped -- those that were correct were now upside down, and vice-versa. Also parts of the box got shifted a bit. This is with ARB shaders BTW. I'll post screen shots when I get the chance. I wonder if there was a reason for the flip code because it does flip some things correctly.
So, I think what we need to do is prevent ourselves from having to do any flipping whatsoever. That's the part that I'm not sure how to do and is the reason for this email.
That I'd agree with. There are certainly things it does wrong. But I think I need to go back studying the code before I make any recommendations. :) I'll try to get back to you later. Jesse
On 25/07/06, Jason Green <jave27(a)gmail.com> wrote:
current renderTarget is not on the current swapchain. The comments in the source say that the upside-downedness is produced by glCopyTexImage, so it sets a flag to flip everything over.
I don't think it's so much that glCopyTexImage flips the texture over, but rather the fact that OpenGL texture coordinates are flipped over the y-axis compared to Direct3D.
So, I think what we need to do is prevent ourselves from having to do any flipping whatsoever. That's the part that I'm not sure how to do and is the reason for this email. Can we load the textures in system memory first, perform a software reversing process, then load that up with glCopyTexImage instead? Will we need to do that type of fixup every time the app locks/unlocks/changes part of the texture? Or, is there a better way?
Can't we just correct the texture coordinates / start writing the surface contents at the bottom? Also, if we use EXT_framebuffer_object instead, we probably won't have to use glCopyTexImage in the first place.
On 7/25/06, H. Verbeet <hverbeet(a)gmail.com> wrote:
Can't we just correct the texture coordinates / start writing the surface contents at the bottom? Also, if we use EXT_framebuffer_object instead, we probably won't have to use glCopyTexImage in the first place.
No idea, but that's the type of answer I was looking for. :-) http://www.winehq.com/hypermail/wine-patches/2004/04/0433.html is the patch which first started this whole concept of flipping to begin with, if that helps.
Am Dienstag 25 Juli 2006 04:21 schrieb Jason Green:
I've spent a couple of days researching the issue of broken/upside-down character/object models in Wine in almost all newer games when you have vertex shaders enabled (Civ4, Half Life 2, Oblivion, Max Payne 2, etc.). I think I've boiled it down to a single case: When "device->renderUpsideDown" is set in the case where vertex shaders are enabled. That flag gets set in device.c:7395 when the current renderTarget is not on the current swapchain. The comments in the source say that the upside-downedness is produced by glCopyTexImage, so it sets a flag to flip everything over.
In the case w/o shaders, there is code in drawprim.c which loads the WORLDVIEW and PROJECTION matrices and then multiplies those matrices by one which inverts the y coordinates when that flag is set. That seems to work in the case without vertex shaders, but when shaders are enabled, they bypass the WORLD, VIEW, and PROJECTION matrices entirely. The shader case was written when only software shaders worked, but that is no longer true. It loads identity matrices and performs the y flip, but that code is entirely irrelevant since the vertex shader doesn't reference those matrices; it only uses constants that are passed by the app, which we can't perform any type of fixup on since we don't know which constants will be used for which calculation.
So, I think what we need to do is prevent ourselves from having to do any flipping whatsoever. That's the part that I'm not sure how to do and is the reason for this email. Can we load the textures in system memory first, perform a software reversing process, then load that up with glCopyTexImage instead? Will we need to do that type of fixup every time the app locks/unlocks/changes part of the texture? Or, is there a better way?
I think I've figured out the problem, it's just the next step of fixing it that I'm unsure of. :-) Can we flip around the y axis in the shader? Or can we flip around the texture coords when drawing from the offscreen texture? If I understand it correctly this only affects offscreen rendering.
On 25/07/06, Stefan Dösinger <stefandoesinger(a)gmx.at> wrote:
Can we flip around the y axis in the shader? Well, that is essentially what Jason's hack does :-)
Or can we flip around the texture coords when drawing from the offscreen texture? If I understand it correctly this only affects offscreen rendering. I was hoping you would know :-) We would have to do some fixups when locking the surface as well, but I don't think that should be a problem.
Am Dienstag 25 Juli 2006 11:54 schrieb H. Verbeet:
On 25/07/06, Stefan Dösinger <stefandoesinger(a)gmx.at> wrote:
Can we flip around the y axis in the shader?
Well, that is essentially what Jason's hack does :-)
Or can we flip around the texture coords when drawing from the offscreen texture? If I understand it correctly this only affects offscreen rendering.
I was hoping you would know :-) We would have to do some fixups when locking the surface as well, but I don't think that should be a problem. Well, I'm not the shader expert in here, but if the y flip only affects offscreen render targets and we can find out in which conditions it needs flipping then we can just create a SFLAG_INVY surface flag(there are quite a lot of them already :-| ). The tricky part is to actually flip it around when drawing because we have to modify the vertices. This could be a case for drawStridedSlow or VBO vertex fixups. Or do the texture matrices help here? (/me reads up the red book)
Surface locking is the least problem, just check the flag and order reversed coordinates from opengl or flip the lines in software.
On 25/07/06, Stefan Dösinger <stefandoesinger(a)gmx.at> wrote:
lot of them already :-| ). The tricky part is to actually flip it around when drawing because we have to modify the vertices. This could be a case for drawStridedSlow or VBO vertex fixups. Or do the texture matrices help here? Well yes, you would modify the texture matrix, rather than the individual coordinates.
Well, we discussed a few different ways of handling this, and here are the results: 1) We can flip the texture we get from the framebuffer in Device_LoadTexture. Getting images from the back framebuffer always are upside down, so we'll need to manually flip it before setting it as the next render target. a) Use glCopyTexSubImage2D() to swap each row b) Use glReadPixels() to copy to a buffer, flip the image, then use glTexImage2D() to write it back. Both of these methods fixed the upside down rendering, but the issue of the disconnected/broken models is still there, although less prevalent now that things look more normal to begin with. I've attached both patches. However, performance takes a pretty serious hit: Max Payne 2 with pixel shaders enabled in High detail mode (there are still some lighting bugs, though): Current git (upside-down): ~65 fps 1a: ~35 fps 1b: ~15 fps Civ4 with pixel shaders enabled Current git (some upside-down stuff and very broken models): 30-50 fps 1a (still some broken models): ~35 fps 1b (still some broken modles): 25-35 fps 2) Try to do some kind of fixup in the shader itself. I'm not entirely sure how to do this other than being creative with the small hack I posted originally. There are no noticable performance losses, but visually, some things are broken (the sky in Tomb Raider Legends as well as Half Life 2 are completely wrong, but Max Payne is about 99% right). Any other suggestions? Also, the models are still broke, so that may be a separate topic and not related like I thought it was.
On 26/07/06, Jason Green <jave27(a)gmail.com> wrote:
2) Try to do some kind of fixup in the shader itself. I'm not entirely sure how to do this other than being creative with the small hack I posted originally. There are no noticable performance losses, It might be possible to fixup the inputs to the sample functions, but it would require regular surfaces to be upside down instead. Not sure if that would be cheaper than flipping the render targets, but regular surfaces typically don't get modified as much. It would also require modifying the texture matrix for the fixed function pipeline.
On 7/25/06, Stefan Dösinger <stefandoesinger(a)gmx.at> wrote:
Can we flip around the y axis in the shader?
It's not quite so easy. At the moment in the case w/o shaders, we multiply the WORLDVIEW and/or PROJECTION matrices by a matrix that flips the y axis on the whole matrix, like so: 1 1 1 1 1 0 0 0 2 2 2 2 0 -1 0 0 3 3 3 3 * 0 0 1 0 4 4 4 4 0 0 0 1 = 1 -1 1 1 2 -2 2 2 3 -3 3 3 4 -4 4 4 However, when you use a shader, you don't use those matrices at all, you pass your own vec4 (4 component float vector) constants into the shader program, and you don't just have 4 of them to use as a matrix, you have as many as the hardware allows (typically 96 or 256 with current hardware). So, we don't know which ones the shader will use as it's MV/P matrix, so we can't perform any type of y-flip at that spot because we might mess up the other constants that the shader needs to perform its calculations. Plus, the same shader can be used for multiple render targets - some upside-down and some not. Flipping the y position in the shader based on the current MVP matrix fixes the issue in some cases, but only when the shader's "effective MVP matrix" is the identity matrix. If the constants that the app passes to the shader differ from the identity matrix, then our y-flip isn't taking any of the other rows y-flip into account, so the calculations are wrong and hence the models are broken.
On 25.07.2006 14:48, Jason Green wrote:
However, when you use a shader, you don't use those matrices at all, you pass your own vec4 (4 component float vector) constants into the shader program, and you don't just have 4 of them to use as a matrix, you have as many as the hardware allows (typically 96 or 256 with current hardware). So, we don't know which ones the shader will use as it's MV/P matrix, so we can't perform any type of y-flip at that spot because we might mess up the other constants that the shader needs to perform its calculations.
However, you do know the register into which the output position will be written by the VP. Could flipping the Y of the output position at the very end of the VP work? -f.r.
On 26/07/06, Frank Richter <frank.richter(a)gmail.com> wrote:
On 25.07.2006 14:48, Jason Green wrote:
However, when you use a shader, you don't use those matrices at all, you pass your own vec4 (4 component float vector) constants into the shader program, and you don't just have 4 of them to use as a matrix, you have as many as the hardware allows (typically 96 or 256 with current hardware). So, we don't know which ones the shader will use as it's MV/P matrix, so we can't perform any type of y-flip at that spot because we might mess up the other constants that the shader needs to perform its calculations.
However, you do know the register into which the output position will be written by the VP. Could flipping the Y of the output position at the very end of the VP work?
-f.r. Well, you can't really do that, since it's not guaranteed the mvp matrix will only be used for position transformations. A shader could use it for eg. lighting calculations.
On 26.07.2006 14:31, H. Verbeet wrote:
However, you do know the register into which the output position will be written by the VP. Could flipping the Y of the output position at the very end of the VP work?
Well, you can't really do that, since it's not guaranteed the mvp matrix will only be used for position transformations. A shader could use it for eg. lighting calculations.
I thought more along the lines of "flip result.position.y", not changing the MVP matrix. -f.r.
On 7/26/06, Frank Richter <frank.richter(a)gmail.com> wrote:
I thought more along the lines of "flip result.position.y", not changing the MVP matrix.
I just sent a patch to wine-patches do just that. I thought that the broken character models would have been a related issue, but that is apparently not the case. Neither of the previous two hacks (which completely flipped the textures at the cost of decreased framerate) had any effect on the character models. My initial hack was on the right track, but it didn't mask the output for y, so it was having issues with other objects that used shaders. So, now it's on to figuring out why the models are disjointed (like this: http://www.cmhousing.net/wine/civ4_brokenmodels.png and this: http://www.cmhousing.net/wine/hl2_models2.png). :-) There are still a few places where I don't think we use shaders that look like they may be rendering upside down, or at least are offset improperly. I'll keep digging into those, too. Thanks, guys!
participants (5)
-
Frank Richter -
H. Verbeet -
Jason Green -
Jesse Allen -
Stefan Dösinger