Are there automated tests that verify that Wine's drawing operations produce the same visual output as in Windows?

[Alexander Almaleh]

While testing the Polygon() function for my bug report -
https://bugs.winehq.org/show_bug.cgi?id=36891, I used a test program that
performs a Polygon() operation on a memory device context, and then saves
it to a bitmap - i.e. a *.bmp file.

I then compared the bitmaps produced under Windows and Wine using a program
I wrote in Qt, that just compares width/height, and then makes a
pixel-by-pixel comparison (and the Polygon() function did indeed produce a
slightly different bitmap).

Since this process can be easily automated, I'm wondering if such a
framework is already being used? If not, I could probably make a sample
test program and image comparison tool, if anyone is interested.

It would be fairly easily to spot all kinds of bugs in the drawing
functions using such a testing system.

[Ken Thomases]

On Jul 13, 2014, at 11:39 AM, Alexander Almaleh  wrote:
> While testing the Polygon() function for my bug report - https://bugs.winehq.org/show_bug.cgi?id=36891, I used a test program that performs a Polygon() operation on a memory device context, and then saves it to a bitmap - i.e. a *.bmp file.
>
> I then compared the bitmaps produced under Windows and Wine using a program I wrote in Qt, that just compares width/height, and then makes a pixel-by-pixel comparison (and the Polygon() function did indeed produce a slightly different bitmap).
>
> Since this process can be easily automated, I'm wondering if such a framework is already being used? If not, I could probably make a sample test program and image comparison tool, if anyone is interested.
>
> It would be fairly easily to spot all kinds of bugs in the drawing functions using such a testing system.

There are such tests.  They were central to the development of the DIB engine.  See dlls/gdi32/tests/dib.c.

The tests run under Wine or on Windows, not both in concert.  They can't directly take output from one and compare to the other.  Also, one doesn't want to store a slew of bitmaps into the tests for pixel-wise comparison.  So, the approach is to take a hash of the bitmap data and compare to a hash of a known-good bitmap.

-Ken

[Alexander Almaleh]

Thanks, I'll look into it.
On Jul 13, 2014 8:09 PM, "Ken Thomases"  wrote:
> On Jul 13, 2014, at 11:39 AM, Alexander Almaleh 
> wrote:
>
> While testing the Polygon() function for my bug report -
> https://bugs.winehq.org/show_bug.cgi?id=36891, I used a test program that
> performs a Polygon() operation on a memory device context, and then saves
> it to a bitmap - i.e. a *.bmp file.
>
> I then compared the bitmaps produced under Windows and Wine using a
> program I wrote in Qt, that just compares width/height, and then makes a
> pixel-by-pixel comparison (and the Polygon() function did indeed produce a
> slightly different bitmap).
>
> Since this process can be easily automated, I'm wondering if such a
> framework is already being used? If not, I could probably make a sample
> test program and image comparison tool, if anyone is interested.
>
> It would be fairly easily to spot all kinds of bugs in the drawing
> functions using such a testing system.
>
> There are such tests.  They were central to the development of the DIB
> engine.  See dlls/gdi32/tests/dib.c.
>
> The tests run under Wine or on Windows, not both in concert.  They can't
> directly take output from one and compare to the other.  Also, one doesn't
> want to store a slew of bitmaps into the tests for pixel-wise comparison.
>  So, the approach is to take a hash of the bitmap data and compare to a
> hash of a known-good bitmap.
>
> -Ken

[Alexander Almaleh]

I see that there is no call to Polygon() anywhere in the tests. I might try
adding a test case for it.

On Sun, Jul 13, 2014 at 9:20 PM, Alexander Almaleh 
wrote:
> Thanks, I'll look into it.
> On Jul 13, 2014 8:09 PM, "Ken Thomases"  wrote:
> > On Jul 13, 2014, at 11:39 AM, Alexander Almaleh 
> > wrote:
> >
> > While testing the Polygon() function for my bug report -
> > https://bugs.winehq.org/show_bug.cgi?id=36891, I used a test program
> > that performs a Polygon() operation on a memory device context, and then
> > saves it to a bitmap - i.e. a *.bmp file.
> >
> > I then compared the bitmaps produced under Windows and Wine using a
> > program I wrote in Qt, that just compares width/height, and then makes a
> > pixel-by-pixel comparison (and the Polygon() function did indeed produce a
> > slightly different bitmap).
> >
> > Since this process can be easily automated, I'm wondering if such a
> > framework is already being used? If not, I could probably make a sample
> > test program and image comparison tool, if anyone is interested.
> >
> > It would be fairly easily to spot all kinds of bugs in the drawing
> > functions using such a testing system.
> >
> > There are such tests.  They were central to the development of the DIB
> > engine.  See dlls/gdi32/tests/dib.c.
> >
> > The tests run under Wine or on Windows, not both in concert.  They can't
> > directly take output from one and compare to the other.  Also, one doesn't
> > want to store a slew of bitmaps into the tests for pixel-wise comparison.
> >  So, the approach is to take a hash of the bitmap data and compare to a
> > hash of a known-good bitmap.
> >
> > -Ken