On Tue, Oct 18, 2011 at 12:08 PM, Henri Verbeet <hverbeet@gmail.com> wrote:
On 18 October 2011 10:45, Damjan Jovanovic <damjan.jov@gmail.com> wrote:
> (especially during "reverse regression testing"), users find it too long and
> technical, and only a small minority of regressions are ever bisected. And
Not true. Even for the regressions that are still open it's currently
276 bisected vs. 99 not bisected, and Alexandre said about 90% of
opened regressions get fixed.


There's currently another 182 regressions that were closed "ABANDONED". Maybe if regression testing was easier and faster, people wouldn't abandon them?
 
> I haven't figured out how to make the binaries available to users. Few users
> can clone a 26 gigabyte repository, and even fewer places can serve that
> much to multiple users. Maybe Git can compress it further? The other idea I
> had is that users should be able to regression test through a GUI tool.
> Maybe the GUI tool can just download and run the +/- 122 MB binary snapshots
> for specific commits, instead of having the entire binary repository
> locally?
>
> Any other ideas? Would you like to see this tool? Can I send an attachment
> with it?
>
Scripts for making old versions compile may be useful, but for the
most part it sounds like you're essentially duplicating ccache.

If you are talking about using compiling with ccache instead of the binary repository, "configure" alone is > 40 seconds, while the average "git bisect" on the binary repository is about 4. If you are talking about using ccache to speed up building the binary repository commit by commit, why, when for most commits "make" takes about 5 seconds and skips all unnecessary compilation anyway?

The other advantage I see is the convenience: instead of having to wait or take breaks while each commit compiles like in normal regression testing, you can have the tool compile you a small set of revisions (eg. the last month) while you are away from your computer, then just quickly test them all in one go when you come back. And you can regression test several applications without repeatedly compiling Wine.