Some conformance tests a bit slow under Valgrind

[Dan Kegel]

Under Valgrind, on my nice fast e7200 system,
the entire suite of tests takes about three hours to run.

The slowest ten percent of the tests take over a third of the runtime.

riched20's editor.c in particular spends way too long
(I think) on test_EM_AUTOURLDETECT.

The min, median, and max times to run a test
are 10 seconds, 19 seconds, and 15 minutes.

15:44.05elapsed riched20.dll editor.c
6:35.94elapsed d3d9.dll visual.c
5:45.43elapsed shell32.dll shlexec.c
3:28.98elapsed kernel32.dll process.c
2:49.85elapsed winmm.dll wave.c
2:32.18elapsed kernel32.dll debugger.c
2:14.41elapsed gdiplus.dll font.c
1:44.54elapsed dsound.dll capture.c
1:29.51elapsed mshtml.dll dom.c
...
0:18.84elapsed shlwapi.dll path.c
0:18.83elapsed rpcrt4.dll cstub.c
...
0:09.92elapsed ntdll.dll path.c
0:09.92elapsed msvcrt.dll data.c

I guess I'm not complaining, but perhaps we could tone
down test_EM_AUTOURLDETECT a bit.

[Alex Villací­s Lasso]

Dan Kegel escribió:
> Under Valgrind, on my nice fast e7200 system,
> the entire suite of tests takes about three hours to run.
>
> The slowest ten percent of the tests take over a third of the runtime.
>
> riched20's editor.c in particular spends way too long
> (I think) on test_EM_AUTOURLDETECT.
>
> The min, median, and max times to run a test
> are 10 seconds, 19 seconds, and 15 minutes.
>
> 15:44.05elapsed riched20.dll editor.c
> 6:35.94elapsed d3d9.dll visual.c
> 5:45.43elapsed shell32.dll shlexec.c
> 3:28.98elapsed kernel32.dll process.c
> 2:49.85elapsed winmm.dll wave.c
> 2:32.18elapsed kernel32.dll debugger.c
> 2:14.41elapsed gdiplus.dll font.c
> 1:44.54elapsed dsound.dll capture.c
> 1:29.51elapsed mshtml.dll dom.c
> ...
> 0:18.84elapsed shlwapi.dll path.c
> 0:18.83elapsed rpcrt4.dll cstub.c
> ...
> 0:09.92elapsed ntdll.dll path.c
> 0:09.92elapsed msvcrt.dll data.c
>
> I guess I'm not complaining, but perhaps we could tone
> down test_EM_AUTOURLDETECT a bit.

I agree that the tests could be toned down. I am open to suggestions on 
what kind of toning down should be implemented. Personally, I would keep 
full testing for WM_SETTEXT, but use just one valid URL for the rest of 
the messages (WM_CHAR, EM_SETTEXTEX, EM_REPLACESEL). All of the known 
delimiters should still be tested in full. Also, there should be a way 
to re-enable full testing for all cases, maybe through an enviroment 
variable.

[Dylan Smith]

On Fri, Jun 27, 2008 at 12:22 PM, Alex Villací­s Lasso <
[e-mail]> wrote:
> Dan Kegel escribió:
> > Under Valgrind, on my nice fast e7200 system,
> > the entire suite of tests takes about three hours to run.
> >
> > The slowest ten percent of the tests take over a third of the runtime.
> >
> > riched20's editor.c in particular spends way too long
> > (I think) on test_EM_AUTOURLDETECT.
> >
> > The min, median, and max times to run a test
> > are 10 seconds, 19 seconds, and 15 minutes.
> >
> > 15:44.05elapsed riched20.dll editor.c
> > 6:35.94elapsed d3d9.dll visual.c
> > 5:45.43elapsed shell32.dll shlexec.c
> > 3:28.98elapsed kernel32.dll process.c
> > 2:49.85elapsed winmm.dll wave.c
> > 2:32.18elapsed kernel32.dll debugger.c
> > 2:14.41elapsed gdiplus.dll font.c
> > 1:44.54elapsed dsound.dll capture.c
> > 1:29.51elapsed mshtml.dll dom.c
> > ...
> > 0:18.84elapsed shlwapi.dll path.c
> > 0:18.83elapsed rpcrt4.dll cstub.c
> > ...
> > 0:09.92elapsed ntdll.dll path.c
> > 0:09.92elapsed msvcrt.dll data.c
> >
> > I guess I'm not complaining, but perhaps we could tone
> > down test_EM_AUTOURLDETECT a bit.
>
> I agree that the tests could be toned down. I am open to suggestions on
> what kind of toning down should be implemented. Personally, I would keep
> full testing for WM_SETTEXT, but use just one valid URL for the rest of
> the messages (WM_CHAR, EM_SETTEXTEX, EM_REPLACESEL). All of the known
> delimiters should still be tested in full. Also, there should be a way
> to re-enable full testing for all cases, maybe through an enviroment
> variable.

Rather than using one URL for all the messages, you could also cycle through
URL after testing them initially.  It would have the same performance of
using the same valid URL, but keeps the variety in the test.

I am attaching a rough patch to show what I mean, and so I can give everyone
an idea of the impact this type of change makes.  The following are times
from calling runtest for the editor tests of riched20.dll before and after
the patch.

Before:
real    1m6.924s
user    0m42.067s
sys    0m7.148s

After:
real    0m11.539s
user    0m3.876s
sys    0m0.956s

Hope this helps.

Because the patch is rough I am not posting it to wine-patches.  Feel free
to use or not use it.

[Dylan Smith]

On Fri, Jun 27, 2008 at 12:43 PM, Dylan Smith  wrote:
> Rather than using one URL for all the messages, you could also cycle through
> URL after testing them initially.  It would have the same performance of
> using the same valid URL, but keeps the variety in the test.

I ended up looking at the test more to fix a valgrind error, and noticed that my
patch should probably only cycle over valid urls instead of all of them.

The valid urls could be moved to the front of the urls array, and then index can
be use the number of valid urls to mod by, rather than the total number of urls.
> I am attaching a rough patch to show what I mean, and so I can give everyone
> an idea of the impact this type of change makes. The following are times
> from calling runtest for the editor tests of riched20.dll before and after
> the patch.
>
> Before:
> real    1m6.924s
> user    0m42.067s
> sys    0m7.148s
>
> After:
> real    0m11.539s
> user    0m3.876s
> sys    0m0.956s

Dan, was this enough of a difference for the time the tests take to run?