Hello! I'd like to get fog working in conjunction with pixel shaders 2.0 or earlier. The problem is that said shaders don't replace the fog stage in Direct3D. Adding the necessary fog calculations to a GLSL fragment shader is trivial, however, this results in a dependency of the shader on the current fog state.
Now, I have thought up three solutions for this problem:
1) Compile two versions of every fragment shader (one with fog and one without) and choose the appropriate one according to the state. Pro: Probably performance wise the cheapest solution Con: Generates more shaders, however, as shader are only compiled on demand, this shouldn't be too bad.
2) Rebuild the shader whenever it is bound in conjunction with a changed fog state. Pro: ? Con: This would be a performance killer as it would potentially cause multiple calls of IWineD3DPixelShaderImpl_CompileShader per frame, as some Programs use the same fragment shader for rendering with and without fog. (ok, I only know this of source based games, but imo that's bad enough)
3) Add Fog calculation to every shader, enabling or disabling it at runtime via a constant factor of 1.0 or 0.0 depending on the fog state which is supplied as a shader constant. Pro: cpu-load wise cheap Con: Adds some more instructions to fragment shader code.
Now, I don't particularly like any of these solutions, so I'd welcome other ideas, comments or suggestions.
Fabian
P.S: I noticed that fog-state management is not yet included in the state table. Should a patch that fixes fog with pixel shaders wait until this is done, or is this of no importance?
Am Montag 12 März 2007 10:35 schrieb Fabian Bieler:
Ok, I am not so much into the whole fog + shader thing, so I am not sure if I understand the problem correcty. The vertex shader writes out fog values, which go into the fog equation. With just a vertex shader only it works, see the dolphin sdk demo.
With pixel Shaders < SM 3.0, the fixed function fog calculation is still performed in Direct3D, but not in opengl? So we have to add a fog emulation to our pixel shaders?
One thing to consider is that there are 2 different fog types in d3d: Fog Calculated per Vertex, and per Fragment fog(or something like this). In gl the difference is just the fog niceness hint(GL_NICEST, GL_FASTEST), but in d3d they seem to be handled very differently with vertex shaders and transformed vertices. And last but not least there is the No-Fog[1] fog which has both vertex and table fog set to NONE but is still quite foggy.
Enabling and disabling fog isn't much of a problem once we have a fog calculation in the shader I think. We can just set fog start, end, exp parameters which effectively eliminate fog.
There will be more problems with selecting the fog equation. It does not apply to the no-fog fog(Linear, start = 255(or 1.0), end = 0), or per vertex fog with vshaders or transformed vertices(simmilar to no-fog fog). With per fragment fog we have linear, exp and exp2. With glsl we can make it a parameter, but for arb shaders we have to hardwire it in the shader. But I think that applications will not switch fog formulas like mad because the fog types won't mix well. So I think the hardwire and recompile approach is suitable.
What we need for the start I think is the ability to recompile the shader. (Destroying the arb / glsl programs, removing it from the linked glsl shader table and reset the shader object). Henri, do you have any patches for that handy or should I take a look? With that I can then also add the signedness fixup for the bump map formats if no suitable extension is available.
If we want to manage multiple compiled shaders with fog on/off then we should extend that to the whole parameter set. We already have sampler type(2D, 3D, cube), texture transform flags(D3DTTF_PROJECTED) and now fog and pixel format coming. My concern is that managing this will get too expensive and grow the amount of linked glsl programs vastly.
- Add Fog calculation to every shader, enabling or disabling it at runtime
via a constant factor of 1.0 or 0.0 depending on the fog state which is supplied as a shader constant. Pro: cpu-load wise cheap Con: Adds some more instructions to fragment shader code.
Somehow you will have to get the fog parameters into the shader. Start and End won't be of much issue, but the formula will be a problem. Not so much with glsl, but arb will be hard.
I think you can just access the fixed function fog states from shaders. But it will use up one or more uniforms.
Fabian
P.S: I noticed that fog-state management is not yet included in the state table. Should a patch that fixes fog with pixel shaders wait until this is done, or is this of no importance?
No, it should be there :-) See STATE_RENDER(WINED3DRS_FOGENABLE), and states linked to it. Or did I miss anything? The apply handler should be state_fog.
[1] I don't know how MS calls it. With the fixed function pipeline fog weightings are stored in the alpha component of the specular vertex color. I think with vshaders it is linear, start = 1.0, end = 0.0 .
On 12/03/07, Stefan Dösinger stefandoesinger@gmx.at wrote:
With per fragment fog we have linear, exp and exp2. With glsl we can make it a parameter, but for arb shaders we have to hardwire it in the shader.
I don't think we really want to use branching for that anyway.
What we need for the start I think is the ability to recompile the shader. (Destroying the arb / glsl programs, removing it from the linked glsl shader table and reset the shader object). Henri, do you have any patches for that handy or should I take a look? With that I can then also add the signedness fixup for the bump map formats if no suitable extension is available.
No, but the hash table functions in utils.c should be generic enough to help at least a little bit with keeping track of the different states.
If we want to manage multiple compiled shaders with fog on/off then we should extend that to the whole parameter set. We already have sampler type(2D, 3D, cube), texture transform flags(D3DTTF_PROJECTED) and now fog and pixel format coming. My concern is that managing this will get too expensive and grow the amount of linked glsl programs vastly.
Up to a certain point. This should only be an issue for pre-2.0 shaders, and MSDN does hint that changing some states (like eg. the sampler type) might cause a shader recompilation, so applications doing that will probably be in trouble on Windows as well.
Somehow you will have to get the fog parameters into the shader. Start and End won't be of much issue, but the formula will be a problem. Not so much with glsl, but arb will be hard.
See my comment above wrt. branching.
I think it's important here to know what exactly is the problem, and under which conditions this happens. It would be a shame if we started adding lots of complexity because of this, only to find out later that it could have been avoided by eg. simply always using per vertex fog when a pixel shader is used.
Am Montag 12 März 2007 13:16 schrieb H. Verbeet:
No, but the hash table functions in utils.c should be generic enough to help at least a little bit with keeping track of the different states.
I am thinking more about completely removing a compiled shader rather than having 2 or more gl shaders for one d3d shader.
If we want to manage multiple compiled shaders with fog on/off then we should extend that to the whole parameter set. We already have sampler type(2D, 3D, cube), texture transform flags(D3DTTF_PROJECTED) and now fog and pixel format coming. My concern is that managing this will get too expensive and grow the amount of linked glsl programs vastly.
Up to a certain point. This should only be an issue for pre-2.0 shaders, and MSDN does hint that changing some states (like eg. the sampler type) might cause a shader recompilation, so applications doing that will probably be in trouble on Windows as well.
Ah, good to know :-)
I think it's important here to know what exactly is the problem, and under which conditions this happens. It would be a shame if we started adding lots of complexity because of this, only to find out later that it could have been avoided by eg. simply always using per vertex fog when a pixel shader is used.
Well, it won't be the always per vertex fog thing because we were doing that up to a patch my Vitaly, but agreed in general :-) . Its test case time :-)
I investigated the issue a bit further. Henri was right, the solution is much simpler than I initially thought. :)
On Monday 12 March 2007 12:56, you wrote:
Am Montag 12 März 2007 10:35 schrieb Fabian Bieler:
With pixel Shaders < SM 3.0, the fixed function fog calculation is still performed in Direct3D, but not in opengl? So we have to add a fog emulation to our pixel shaders?
That's how I see the problem.
One thing to consider is that there are 2 different fog types in d3d: Fog Calculated per Vertex, and per Fragment fog(or something like this). In gl the difference is just the fog niceness hint(GL_NICEST, GL_FASTEST), but in d3d they seem to be handled very differently with vertex shaders and transformed vertices. And last but not least there is the No-Fog[1] fog which has both vertex and table fog set to NONE but is still quite foggy.
As I see it, the No-Fog fog is a vertex fog where the fog weighting parameter for every vertex is calculated by the application on the cpu rather than by geometry pipeline.
Anyhow from MSDN: "When using a vertex shader, you must use vertex fog. This is accomplished by using the vertex shader to write the per-vertex fog intensity to the oFog register. After the pixel shader completes, the oFog data is used to linearly interpolate with the fog color. This intensity is not available in a pixel shader."
This is what we have to do, imo: Take the value from the oFog register, and store it in gl_FogFragCoord (which is currently already done). At the end of the fragment shader, we have to clamp gl_FogFragCoord to [0,1] and used it to mix the fragment color with the fog color, if fog is enabled. An Idea would be to use shader constants as the clamping parameters and to clamp gl_FogFragCoord to [1,1] if fogging is disabled and [0,1] if fogging is enabled.
On a sidenote: MSDN also states: "An application can implement fog with a vertex shader, and pixel fog simultaneously if desired." However, I think this is rather a corner case. Which application would want to use two fogs simultaneously?
P.S: I noticed that fog-state management is not yet included in the state table. Should a patch that fixes fog with pixel shaders wait until this is done, or is this of no importance?
No, it should be there :-) See STATE_RENDER(WINED3DRS_FOGENABLE), and states linked to it. Or did I miss anything? The apply handler should be state_fog.
sorry, I misunderstood a comment in the code.
On 12/03/07, Fabian Bieler der.fabe@gmx.net wrote:
This is what we have to do, imo: Take the value from the oFog register, and store it in gl_FogFragCoord (which is currently already done). At the end of the fragment shader, we have to clamp gl_FogFragCoord to [0,1] and used it to mix the fragment color with the fog color, if fog is enabled. An Idea would be to use shader constants as the clamping parameters and to clamp gl_FogFragCoord to [1,1] if fogging is disabled and [0,1] if fogging is enabled.
I think that if fog is disabled, we should be getting corresponding values from vertex processing. Also, at least for the vertex shader case we already clamp fog values in the vertex shader, so unless fixed function can pass us values outside [0,1] we probably can get away with not clamping at all.
I think that if fog is disabled, we should be getting corresponding values from vertex processing. Also, at least for the vertex shader case we already clamp fog values in the vertex shader, so unless fixed function can pass us values outside [0,1] we probably can get away with not clamping at all.
Good question.
With the precalculated No-Fog fog I pass values from [255,0] in the vertex attribs. GL fogstart = 255, end = 0, but that is bypassed with pixel shaders.
Otherwise the Z value is used, which can be in any range. So yes, I think the fixed function pipeline can pass any range in. Also we should check how table fog really behaves with shaders.
On 12/03/07, Stefan Dösinger stefandoesinger@gmx.at wrote:
With the precalculated No-Fog fog I pass values from [255,0] in the vertex attribs. GL fogstart = 255, end = 0, but that is bypassed with pixel shaders.
Otherwise the Z value is used, which can be in any range. So yes, I think the fixed function pipeline can pass any range in. Also we should check how table fog really behaves with shaders.
Well, that's what you pass to vertex processing. But I think that what comes out of vertex processing is already clamped in OpenGL. My reason for thinking that is that in OpenGL we have to clamp the fog coordinates in the vertex shader because fixed function fragment processing doesn't seem to do any clamping.
As I see it, the No-Fog fog is a vertex fog where the fog weighting parameter for every vertex is calculated by the application on the cpu rather than by geometry pipeline.
Yup. And it is also used if vertex fog is enabled but the app draws already transformed vertices.
Anyhow from MSDN: "When using a vertex shader, you must use vertex fog.
Prince of Persia: The sands of time does exactly the oposite. So I am not sure if we can trust that line.
On a sidenote: MSDN also states: "An application can implement fog with a vertex shader, and pixel fog simultaneously if desired."
How does that match with the statement quoted above?
However, I think this is rather a corner case. Which application would want to use two fogs simultaneously?
I think they are called DirectX apps :-)
P.S: I noticed that fog-state management is not yet included in the state table. Should a patch that fixes fog with pixel shaders wait until this is done, or is this of no importance?
No, it should be there :-) See STATE_RENDER(WINED3DRS_FOGENABLE), and states linked to it. Or did I miss anything? The apply handler should be state_fog.
sorry, I misunderstood a comment in the code.
Which comment was that? It could be outdated, or whatever it is confusing, so the comment should be fixed.
-
Fabian Bieler -
H. Verbeet -
Stefan Dösinger