Re: GSoC Idea: Tools - Winetest Scripting Interface

[Zhenbo Li]

Hello, Henri

Thank you for your reply.

2016-03-04 20:26 GMT+08:00 Henri Verbeet :
> On 4 March 2016 at 09:47, Zhenbo Li  wrote:
> > Alice is developing graphics driver, and wine's test for
> > d3d(8,9,10,11) would be a great test case. She was following this
> > workflow.
> > - Edit graphics driver
> > - cd /path/to/wine; cd dlls/d3dx9_36/tests; make test;
> > - cd /path/to/wine; cd dlls/d3d9/tests; make test;
> >
> > (test other graphics related dlls)
> > 4. Edit graphics driver again.
> > 5. Run wine tests again, and compare them manually.
> >
> > With improved wine test suite, her workflow can be
> > - Edit graphics driver
> > - wine_test_suite --test_dll d3dx9_36 d3d9 d3d8 etc. --output result.txt
> > - Edit graphics driver again
> > - wine_test_suite --test_dll d3dx9_36 d3d9 d3d8 etc. --compare result.txt
> >
> > And the script will tell Alice whether she had introduced, or fixed a bug.
>
> winetest can do this.

Do you mean setting the filter in winetest?
> > B.
> > Bob is writing a patch for wine headers. After modification, he typed
> > 'make test' and left the computer for lunch. When he came back, he
> > found that dlls/comdlg32/tests failed(it always happens on my PC), and
> > the make script stopped. He needed to spend another hour to test the
> > remaining components.
> > With the script, he could type
> > wine_test_suite --test_all --output result.txt
> > And Bob can get everything he wants after his lunch.
>
> winetest can do this.

Thank you for mentioning that.
> > C.
> > Charlie is working on wininet. He was afraid that his patch might
> > broke other dlls which depend on wininet. With this script, he can run
> > wine_test_suite --group net
> > And all cases related to net can be tested(net is pre-configured in the script)
>
> It might be more interesting if the test suite could figure out the
> tests potentially affected by a range of commits on its own. I.e., if
> I make a change to wined3d that potentially affects d3drm and d2d1,
> among others.

This sounds good. A straight approach is to scan Makefile.in, and
generate a dependence tree from IMPORT and DELAYIMPORTS. However, I'm
wondering how to handle the dynamic dependencies. For example, in
dlls/mshtml/xmlhttprequest.c, we have
    hres = CoCreateInstance(&CLSID_DOMDocument, NULL,
CLSCTX_INPROC_SERVER, &IID_IXMLDOMDocument, (void**)&xmldoc);
And the development for msxml really affected mshtml.dll's behavior.

I did a roughly estimation
$ git grep CoCreateInstance|grep -v spec|grep -v tests| wc -l
162

Maybe we can write a dependency detector, and add some exceptions for
critical dynamic dependencies?

Thank you.