Automatic Regression Testing

[Paul Millar]

I was sitting in a sitting room, 
  Toying with some toys,
  When from a room marked Gruesome,
  There came a gruesome noise.

I've been "toying" with an automatic regression testing system over the
past month or so. The results can be seen at:

http://www.astro.gla.ac.uk/users/paulm/WRT/

Currently its only using the Perl framework, which doesn't test too much
at the moment. Hopefully this will change soon.

Comments gratefully received (and any criticisms too, I guess;)

Cheers,

Paul.

[Andriy Palamarchuk]

--- Paul Millar  wrote:
> I've been "toying" with an automatic regression
> testing system over the
> past month or so. The results can be seen at:
>
> http://www.astro.gla.ac.uk/users/paulm/WRT/
>
> Currently its only using the Perl framework, which
> doesn't test too much
> at the moment. Hopefully this will change soon.

Looks good. I don't know if we need this information
online. However, having the automated tests and
mailing wine-devel in case something breaks is,
probably a good idea.

Andriy Palamarchuk

__________________________________________________
Do You Yahoo!?
Yahoo! Sports - live college hoops coverage
http://sports.yahoo.com/

[Paul Millar]

On Wed, 20 Mar 2002, Andriy Palamarchuk wrote:
> --- Paul Millar  wrote:
> > http://www.astro.gla.ac.uk/users/paulm/WRT/
>
> Looks good. I don't know if we need this information
> online.

True.

We could get away with just an email when something broke, but the web
interface is supposed to provide a more elegant way of discovering what
went wrong.

There's quite a lot of information in the raw data (it can easily be over
300k). Obviously I couldn't send that to wine-devel: I would have to trim
it down. The web interface should allow people to select the important
information and ignore the rest.

There's also links to cvsweb and patch information, so figuring out which 
change caused the regression should be easy (fixing it might not be, 
though ;)
> However, having the automated tests and
> mailing wine-devel in case something breaks is,
> probably a good idea.

Yep, emailing wine-devel sounds like a good idea.

What information would be useful?

I think the minimum is a list of applied patches, the tests that broke and
a link to the buildinfo web page.

Paul.

[Andriy Palamarchuk]

--- Paul Millar  wrote:
> > However, having the automated tests and
> > mailing wine-devel in case something breaks is,
> > probably a good idea.
>
> Yep, emailing wine-devel sounds like a good idea.
>
> What information would be useful?
>
> I think the minimum is a list of applied patches,
> the tests that broke and
> a link to the buildinfo web page.

Right, and the email should be sent only when
something is broken.

Andriy

__________________________________________________
Do You Yahoo!?
Yahoo! Sports - live college hoops coverage
http://sports.yahoo.com/