- stability problems The buildbot cluster is not ready for prime time yet. My ubuntu 11.04-based slaves tend to lock up within a day with either an OOM failure, an X livelock, or something else. So I brought an ubuntu 10.04.3 slave online and moved the buildmaster off onto its own machine; let's see what breaks next.
- Slave speed Here is the complete build-and-test time (and the build time alone) for four CPUs: 54 (48) celeron @ 2.80 GHz (headless, so it's not running all tests) 17 (10) E8400 @ 3.00GHz 14 (6) Q9300 @ 2.50GHz 12 (4) i7 920 @ 2.67GHz
I will probably stick with slaves that are core 2 duo or better, so patches can get fast feedback.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi,
Is it possible to set up a slave that e.g. runs only the d3d tests? I have some older machines with older GPUs that are still worth covering(Radeon X1600, Geforce 7, maybe even the Radeon 9000 mobility), but I am not sure if the machines as a whole are powerful enough to run all the tests. I think there isn't really a need to run the mostly platform independent tests like msi and shell32 everywhere.
Stefan
Am 27.08.2011 um 21:00 schrieb Dan Kegel:
- stability problems
The buildbot cluster is not ready for prime time yet. My ubuntu 11.04-based slaves tend to lock up within a day with either an OOM failure, an X livelock, or something else. So I brought an ubuntu 10.04.3 slave online and moved the buildmaster off onto its own machine; let's see what breaks next.
- Slave speed
Here is the complete build-and-test time (and the build time alone) for four CPUs: 54 (48) celeron @ 2.80 GHz (headless, so it's not running all tests) 17 (10) E8400 @ 3.00GHz 14 (6) Q9300 @ 2.50GHz 12 (4) i7 920 @ 2.67GHz
I will probably stick with slaves that are core 2 duo or better, so patches can get fast feedback.
On Sun, Aug 28, 2011 at 2:16 AM, Stefan Dösinger stefandoesinger@gmx.at wrote:
Is it possible to set up a slave that e.g. runs only the d3d tests? I have some older machines with older GPUs that are still worth covering(Radeon X1600, Geforce 7, maybe even the Radeon 9000 mobility), but I am not sure if the machines as a whole are powerful enough to run all the tests. I think there isn't really a need to run the mostly platform independent tests like msi and shell32 everywhere.
Yes, I can set up a builder type that only triggers on d3d-related changes and only runs d3d tests. - Dan
On Sunday 28 August 2011 17:52:21 Dan Kegel wrote:
On Sun, Aug 28, 2011 at 2:16 AM, Stefan Dösinger stefandoesinger@gmx.at
wrote:
Is it possible to set up a slave that e.g. runs only the d3d tests? I have some older machines with older GPUs that are still worth covering(Radeon X1600, Geforce 7, maybe even the Radeon 9000 mobility), but I am not sure if the machines as a whole are powerful enough to run all the tests. I think there isn't really a need to run the mostly platform independent tests like msi and shell32 everywhere.
Yes, I can set up a builder type that only triggers on d3d-related changes and only runs d3d tests.
I got the r200 machine updated to a point where I could run buildbot, but I realized that the days when this GPU/driver passed the tests are long gone. A few of those failures are low hanging fruits, at least as far writing bug reports is concerned, but others need more debugging.
The kinds of failures(vertex shaders are completely broken, for example) make me doubt that we have more than a handful of users using this sort of hardware, otherwise we'd have more bug reports about them.
On the other hand I think having a hardware with "strange" limits in the testbot system can help to avoid getting bad assumptions about those limits into the code, so I'll spend a day or two trying to isolate the issues and at least file bug reports for Mesa and Wine, and actually running the buildbot to see if this ancient machine can at least keep up with the d3d and d3dx patches.
On Wed, Aug 31, 2011 at 3:53 PM, Stefan Dösinger stefandoesinger@gmx.at wrote:
On the other hand I think having a hardware with "strange" limits in the testbot system can help to avoid getting bad assumptions about those limits into the code, so I'll spend a day or two trying to isolate the issues and at least file bug reports for Mesa and Wine, and actually running the buildbot to see if this ancient machine can at least keep up with the d3d and d3dx patches.
it'd probably be a better use of time to bring up a modern ati system?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 01.09.2011 um 01:12 schrieb Dan Kegel:
it'd probably be a better use of time to bring up a modern ati system?
If I had one I could use as-is, yes. My computer with a modern AMD GPU is in my working and sleeping room, where I'd like to keep it for coding, but I can't keep it on all the time.
The iMac would be an idea, but there's still a rather nasty issue with the lack of xrandr and xvidmode support on Apple's X server, and a wine hack that is needed with CrossOver's X server. If I read your scripts correctly you run the tests in a virtual desktop, in which case this wouldn't be an issue.
If on the other hand you have an system with a AMD GPU(using fglrx or the open source driver doesn't matter) and have some issues setting it up(e.g. the d3d tests crashing the X server) I can look into those issues and help you get it running on your side.
On 1 September 2011 09:19, Stefan Dösinger stefandoesinger@gmx.at wrote:
The iMac would be an idea, but there's still a rather nasty issue with the lack of xrandr and xvidmode support on Apple's X server, and a wine hack that is needed with CrossOver's X server. If I read your scripts correctly you run the tests in a virtual desktop, in which case this wouldn't be an issue.
Note that virtual desktop breaks some (iirc user32) tests, unless that's something that was fixed recently.
If on the other hand you have an system with a AMD GPU(using fglrx or the open source driver doesn't matter)
Did the tests start passing on fglrx then?
On Thu, Sep 1, 2011 at 4:04 AM, Henri Verbeet hverbeet@gmail.com wrote:
Note that virtual desktop breaks some (iirc user32) tests, unless that's something that was fixed recently.
I work around it. There were only a couple. - Dan
On Thursday 01 September 2011 13:04:34 Henri Verbeet wrote:
Did the tests start passing on fglrx then?
As far as I know they don't. But do they have to?
If I remember correctly Dan's previous patchwatcher scripts kept track of tests that were failing, so they didn't require a fully successful test run. The important thing was that the tests succeeded or failed reliably.
The new setup looks much less sophisticated in that regard, at very least it can't check single ok() lines(or I haven't found the code that does this)
On Thu, Sep 1, 2011 at 8:38 AM, Stefan Dösinger stefandoesinger@gmx.at wrote:
On Thursday 01 September 2011 13:04:34 Henri Verbeet wrote:
Did the tests start passing on fglrx then?
As far as I know they don't. But do they have to?
If I remember correctly Dan's previous patchwatcher scripts kept track of tests that were failing, so they didn't require a fully successful test run. The important thing was that the tests succeeded or failed reliably.
The new setup looks much less sophisticated in that regard
Right - I have not yet forward ported the code that tracks known failures. It hasn't been needed so far -- I've gotten away with a simple blacklist -- but once I start doing things like valgrinding, checking compiler warnings, or running on ATI I'll probably need to bring it back. - Dan
On 1 September 2011 17:38, Stefan Dösinger stefandoesinger@gmx.at wrote:
On Thursday 01 September 2011 13:04:34 Henri Verbeet wrote:
Did the tests start passing on fglrx then?
As far as I know they don't. But do they have to?
If running those tests is the point of that machine, I'd say so. With a known broken configuration, you risk just running into the same known bugs every time you change something or add a new test, wasting everyone's time. An important consideration here is that apparently upstream isn't interested in fixing this kind of bug, and there's no way for us to fix them ourselves.
Fwiw, the d3d8 and d3d9 device tests fail, no idea why, the d3d9 query test fails(this is something we can fix). The ddraw, d3d8 and d3d9 visual tests can't read back the backbuffer, so they refuse to run.
On Thursday 01 September 2011 18:18:11 Henri Verbeet wrote:
On 1 September 2011 17:38, Stefan Dösinger stefandoesinger@gmx.at wrote:
On Thursday 01 September 2011 13:04:34 Henri Verbeet wrote:
Did the tests start passing on fglrx then?
As far as I know they don't. But do they have to?
If running those tests is the point of that machine, I'd say so. With a known broken configuration, you risk just running into the same known bugs every time you change something or add a new test, wasting everyone's time. An important consideration here is that apparently upstream isn't interested in fixing this kind of bug, and there's no way for us to fix them ourselves.
On Thursday 01 September 2011 18:46:58 Stefan Dösinger wrote:
Fwiw, the d3d8 and d3d9 device tests fail, no idea why, the d3d9 query test fails(this is something we can fix). The ddraw, d3d8 and d3d9 visual tests can't read back the backbuffer, so they refuse to run.
This crash happens in the CreateDevice call that uses the D3DCREATE_FPU_PRESERVE flag. The exception is a division by zero exception in the driver.
I had a quick look at the D3DCREATE_FPU_PRESERVE code: shouldn't we do something like ddraw where we set the FPU control word in every method and restore it afterwards? I think we can't blame the driver for crashing if we hand it over the FPU in a messed up state. It probably needs some more test, e.g. what do methods other than CreateDevice do when FPU_PRESERVE is not set and the FPU is in some non-standard mode.
No idea yet why the visual tests don't work.
On 1 September 2011 19:00, Stefan Dösinger stefan@codeweavers.com wrote:
I had a quick look at the D3DCREATE_FPU_PRESERVE code: shouldn't we do something like ddraw where we set the FPU control word in every method and restore it afterwards? I think we can't blame the driver for crashing if we hand it over the FPU in a messed up state. It probably needs some more test, e.g. what do methods other than CreateDevice do when FPU_PRESERVE is not set and the FPU is in some non-standard mode.
No idea yet why the visual tests don't work.
What it comes down to is this: Both Mesa and NVIDIA can manage to somehow make this work, fglrx isn't interested in making it work. That's their good right of course, but the consequence is that I from my side am not particularly interested in making things work with fglrx. I'd much rather spend that time working with the r300g and r600g people on making those drivers as good as possible.
Now if you can justify working on this, either towards your employer or towards yourself in your own time, I'm certainly not going to stop you. All I care about in that regard is that any resulting patches are up to standards. However, what I want to avoid is the situation where I'm spending time debugging fglrx because the buildbot has test failures there.
Looking at this from a slightly different angle, I think that if you expect people to take buildbot results seriously, you should run it on a configuration that's known to be solid, so that people can be confident any failures are actual issues with the patch, instead of e.g. fglrx or Ubuntu breaking something again.
On 09/01/2011 10:50 AM, Henri Verbeet wrote:
Looking at this from a slightly different angle, I think that if you expect people to take buildbot results seriously, you should run it on a configuration that's known to be solid, so that people can be confident any failures are actual issues with the patch, instead of e.g. fglrx or Ubuntu breaking something again.
Finding out when Ubuntu breaks something is important too, and indeed one reason I started the daily Ubuntu builds and will be doing them with the development release.
A testbot can't really know if it was Ubuntu breaking something or us breaking something that was only exposed on Ubuntu. We can make prior assumptions for individual machines though, such as it's usually fglrx's fault, and adjust our behavior/failure warnings accordingly.
Thanks, Scott Ritchie
On 08/27/2011 12:00 PM, Dan Kegel wrote:
- stability problems
The buildbot cluster is not ready for prime time yet. My ubuntu 11.04-based slaves tend to lock up within a day with either an OOM failure, an X livelock, or something else. So I brought an ubuntu 10.04.3 slave online and moved the buildmaster off onto its own machine; let's see what breaks next.
- Slave speed
Here is the complete build-and-test time (and the build time alone) for four CPUs: 54 (48) celeron @ 2.80 GHz (headless, so it's not running all tests) 17 (10) E8400 @ 3.00GHz 14 (6) Q9300 @ 2.50GHz 12 (4) i7 920 @ 2.67GHz
I will probably stick with slaves that are core 2 duo or better, so patches can get fast feedback.
I can get to work on backporting all the things Wine needs to the buildbot for 10.04, btw.
Thanks, Scott Ritchie
-
Dan Kegel -
Henri Verbeet -
Scott Ritchie -
Stefan Dösinger -
Stefan Dösinger