Proposal: winetest 'context'

[Mark Jansen]

Hello,

Often when testing something, helper functions are introduced that are
called with different data (for example when iterating over tables).

Attached patch (winetest_set_context.diff) is a proposal that allows
tests to specify a 'context', that will be printed in the failure
message.
This way, the actual test code does not have to prefix every case with
a trace, or include the data in every test macro.
The patch is written so that tests do not have to use this mechanism,
so there is no need to convert all tests at once.
Second attachment (winetest_ctx_example.diff) shows how an existing
test could be converted to use this new technique.

Regards,

Mark Jansen

[Józef Kucia]

On Tue, Feb 14, 2017 at 5:04 PM, Mark Jansen  wrote:
> Hello,
>
> Often when testing something, helper functions are introduced that are
> called with different data (for example when iterating over tables).
>
> Attached patch (winetest_set_context.diff) is a proposal that allows
> tests to specify a 'context', that will be printed in the failure
> message.
> This way, the actual test code does not have to prefix every case with
> a trace, or include the data in every test macro.
> The patch is written so that tests do not have to use this mechanism,
> so there is no need to convert all tests at once.
> Second attachment (winetest_ctx_example.diff) shows how an existing
> test could be converted to use this new technique.

For what it's worth, if we had a mechanism to set a context for test
messages, I would gladly use it in d3d11 tests.

Cheers,
Józef

[Sebastian Lackner]

On 25.02.2017 13:34, Józef Kucia wrote:
> On Tue, Feb 14, 2017 at 5:04 PM, Mark Jansen  wrote:
> > Hello,
> >
> > Often when testing something, helper functions are introduced that are
> > called with different data (for example when iterating over tables).
> >
> > Attached patch (winetest_set_context.diff) is a proposal that allows
> > tests to specify a 'context', that will be printed in the failure
> > message.
> > This way, the actual test code does not have to prefix every case with
> > a trace, or include the data in every test macro.
> > The patch is written so that tests do not have to use this mechanism,
> > so there is no need to convert all tests at once.
> > Second attachment (winetest_ctx_example.diff) shows how an existing
> > test could be converted to use this new technique.
>
> For what it's worth, if we had a mechanism to set a context for test
> messages, I would gladly use it in d3d11 tests.
>
> Cheers,
> Józef

I would also like to see such a feature, however I'm not really happy with
a single context string which has to be set and reset manually. So, here is
an alternative proposal which allows to use a block-syntax like:

--- snip ---
with_context("i = %d, j = %d", i, j)
{
    // here some tests
}
--- snip ---

Blocks can also be nested, and its easy to combine them with a loop:

--- snip ---
for (i = 0; i < 100; i++) with_context("test %d", i)
{
     // here some tests (don't use break!)
}
--- snip ---

The only disadvantage is that we need variadic macros (__VA_ARGS__), but it
looks like they are already used in a bunch of other tests, so thats probably
not a real problem. Please let me know what you think about this method, and
also if you have suggestions for a better name.

Best regards,
Sebastian