Re: [PATCH 1/5] wined3d: Clamp fog coordinate in the vertex shader.

[Stefan Dösinger]

Am Donnerstag, 8. März 2012, 18:22:15 schrieb Matteo Bruni:
> -
>            shader_addline(buffer, "gl_FogFragCoord = OUT[%u].%c;\n",
>
> i, reg_mask[1]);
> -
>            shader_addline(buffer, "gl_FogFragCoord =
>
> clamp(OUT[%u].%c, 0.0, 1.0);\n", i, reg_mask[1]);

Is it correct to clamp the fog coord in the vertex shader, or should this be 
done in the pixel shader? The tests do not answer this question because they 
write the same fog coordinate for each vertex.

Imagine for example two lines. In each of those lines, one vertex has oFog = 
0.0. The other vertex has oFog = 1.1 in one line, and oFog = 100 in the other. 
If the value is clamped per vertex, those lines will look the same. If it's 
clamped per pixel, the latter line will have much steeper gradient

Also, what happens to shaders that do not write oFog, but use oPos.z, 
especially in the situations where the near and far clipping planes are 
disabled?

[PATCH 1/5] wined3d: Clamp fog coordinate in the vertex shader.

[Matteo Bruni]

Il 08 marzo 2012 22:10, Stefan Dösinger  ha scritto:
> Am Donnerstag, 8. März 2012, 18:22:15 schrieb Matteo Bruni:
> > - shader_addline(buffer, "gl_FogFragCoord = OUT[%u].%c;\n",
> >
> > i, reg_mask[1]);
> > - shader_addline(buffer, "gl_FogFragCoord =
> >
> > clamp(OUT[%u].%c, 0.0, 1.0);\n", i, reg_mask[1]);
>
> Is it correct to clamp the fog coord in the vertex shader, or should this be
> done in the pixel shader? The tests do not answer this question because they
> write the same fog coordinate for each vertex.
>
> Imagine for example two lines. In each of those lines, one vertex has oFog =
> 0.0. The other vertex has oFog = 1.1 in one line, and oFog = 100 in the other.
> If the value is clamped per vertex, those lines will look the same. If it's
> clamped per pixel, the latter line will have much steeper gradient

I think I wrote a test at some point which shows that it is actually
per-vertex. But I may be mistaken and such a test should be in the
testsuite anyway, so I'll add it.
> Also, what happens to shaders that do not write oFog, but use oPos.z,
> especially in the situations where the near and far clipping planes are
> disabled?

That's another good point, I don't think I tested that (that case
isn't used by either Soul Reaver or the Sands of Time, the two games I
was looking into at the time).
I'll update the patches accordingly.