new d3d9/device.ok test always fails here, but not a regression?
Happy New Year's Eve, everyone! 002447357c2b1feca5ef2649429fc55f70238901 added a IDirect3DDevice9::SetCursorPosition() test to d3d9. That test never worked here, I think. I filed http://bugs.winehq.org/show_bug.cgi?id=29094 for it, and marked it as a regression (since tests/device.c used to pass, but fails here after that commit). Henri removed the regression keyword and sha1sum without comment. What's the reasoning there? Is Henri saying there's something wrong with my test setup (stock ubuntu), or are new tests that cause 'make test' to fail not considered regressions? Or something else?
On Sat, Dec 31, 2011 at 11:43, Dan Kegel <dank(a)kegel.com> wrote:
Happy New Year's Eve, everyone!
002447357c2b1feca5ef2649429fc55f70238901 added a IDirect3DDevice9::SetCursorPosition() test to d3d9. That test never worked here, I think. I filed http://bugs.winehq.org/show_bug.cgi?id=29094 for it, and marked it as a regression (since tests/device.c used to pass, but fails here after that commit).
Henri removed the regression keyword and sha1sum without comment.
What's the reasoning there? Is Henri saying there's something wrong with my test setup (stock ubuntu), or are new tests that cause 'make test' to fail not considered regressions? Or something else?
That _particular_ test never worked, so it's not a regression. A code change in d3d itself that broke a previously working test/application would be a valid regression. -- -Austin
On Sun, Jan 1, 2012 at 6:32 PM, Austin English <austinenglish(a)gmail.com> wrote:
http://bugs.winehq.org/show_bug.cgi?id=29094 Henri removed the regression keyword and sha1sum without comment.
What's the reasoning there? Is Henri saying there's something wrong with my test setup (stock ubuntu), or are new tests that cause 'make test' to fail not considered regressions? Or something else?
That _particular_ test never worked, so it's not a regression. A code change in d3d itself that broke a previously working test/application would be a valid regression.
IMHO a code change in a d3d test that broke a previously working "make test" also counts as a regression. The test suite is *important*. Breakage in the test suite is not trivial. - Dan
Dan Kegel <dank(a)kegel.com> writes:
On Sun, Jan 1, 2012 at 6:32 PM, Austin English <austinenglish(a)gmail.com> wrote:
That _particular_ test never worked, so it's not a regression. A code change in d3d itself that broke a previously working test/application would be a valid regression.
IMHO a code change in a d3d test that broke a previously working "make test" also counts as a regression.
The test suite is *important*. Breakage in the test suite is not trivial.
Obviously it's important, and we have an importance field in bugzilla for that. But the regression keyword has a specific meaning: there has to be a piece of code that worked and then got broken. Bugs in newly added code cannot be regressions. Similarly, if we add a dll and some app starts using it and breaks, technically that's not a regression even though the behavior got worse, because it has always been broken, it just wasn't exercised before. -- Alexandre Julliard julliard(a)winehq.org
On Mon, Jan 2, 2012 at 2:58 AM, Alexandre Julliard <julliard(a)winehq.org> wrote:
Obviously it's important, and we have an importance field in bugzilla for that. But the regression keyword has a specific meaning: there has to be a piece of code that worked and then got broken. Bugs in newly added code cannot be regressions.
Similarly, if we add a dll and some app starts using it and breaks, technically that's not a regression even though the behavior got worse, because it has always been broken, it just wasn't exercised before.
OK, thanks for the clarification. If an app stops working because some missing feature is added to an existing DLL, it should not be tagged as a regression even though it is from the app's point of view, right? (Thinking of the installers for Photoshop CS3 and Visual Studio 2005.) I wonder if we need a separate keyword for that, like 'appregression'. - Dan
* On Mon, 2 Jan 2012, Dan Kegel wrote:
If an app stops working because some missing feature is added to an existing DLL, it should not be tagged as a regression even though it is from the app's point of view, right? (Thinking of the installers for Photoshop CS3 and Visual Studio 2005.) I wonder if we need a separate keyword for that, like 'appregression'.
I would call it 'newfeature' or even 'weaknewfeature' to avoid similarity between sound of 'regression' and 'appregression'. S.
On Tue, Jan 3, 2012 at 01:10, Saulius Krasuckas <saulius2(a)ar.fi.lt> wrote:
* On Mon, 2 Jan 2012, Dan Kegel wrote:
If an app stops working because some missing feature is added to an existing DLL, it should not be tagged as a regression even though it is from the app's point of view, right? (Thinking of the installers for Photoshop CS3 and Visual Studio 2005.) I wonder if we need a separate keyword for that, like 'appregression'.
I would call it 'newfeature' or even 'weaknewfeature' to avoid similarity between sound of 'regression' and 'appregression'.
I don't think it's a common enough occurrence to warrant a keyword. -- -Austin
participants (4)
-
Alexandre Julliard -
Austin English -
Dan Kegel -
Saulius Krasuckas