Zebediah Figura z.figura12@gmail.com writes:
The ultimate goal, of course, is that we finally get a way to run testbot patches on [ideally multiple] Linux machines. Nobody needs me to tell them how important this is, but of course it's also important that we start from a clean slate, since these tests are going to be run for *every* patch, not just those that modify the tests. So while Linux is in a better state than some Windows versions at this point, it seems to me that we really do want a wholly green machine.
While we do want a green machine, note that this is not required for starting to run the tests on Linux. The testbot is able to filter known errors out, and we can ignore the spurious ones like we do for Windows tests.
Also along these lines: presumably setting up the testbot to run linux tests is going to require a fair amount of work, and that work is going to be implied to be François' responsibility. I'd just like to say that I'm willing to help as much as I can, though.
It would be great if you could work on that. As far as I'm concerned this should be the top priority for the testbot, but it has not been getting any traction for years.
This isn't, I assert, something wrong with my setup. The D3D team's obviously admirable predisposition toward verbosity, and the need to print a large number of floating-point values, combined with a total of 2167 tests marked todo, has taken over almost one-sixth of the report. But even with d3d11 removed, the report just barely sneaks in under the limit, at 1475812 bytes. One might blame an excess of trace lines, as last time, but in all reality, I am inclined to think that 1.5 MB is no longer a reasonable limit for a test report. If there is an inherent limitation brought on by server storage space or processing power, well, so be it, but if not we probably will want to either raise it—or otherwise start investing energy into getting rid of existing todos.
While we can bump the limit a little, we still need to invest energy into reducing verbosity. If we instead increase the limit every time some test runs over, we'll reach 100MB in a couple of years, and then we'll be in serious trouble.