Regression testing with Wine is a pain in the butthole.
For very bad reasons: Stuff that is simple to fix, but haven't been.
I'd like to help improve that situation.
Example: ========
I want to find which patch ruins an application. The application, according to one note, worked in 2003. I decide to try and see how it really runs with the 2003 version of Wine mentioned in the note.
I then spend several days pulling various releases, just trying to compile them. They all fail miserably, because some fix required to compile correctly with newer Linux versions is /missing/.
For example, - any Wine before 2004-01-02 won't work because it won't compile against newer ALSA versions. - any Wine before 2003-12-01 needs patches to the build system before 'make depend' will even run.
I've been at it for a week and I've managed to fix the above two. _They're really simple things, but they take a lot of time to figure out._
Currently I'm working with two other problems: - Wine fails to correctly invoke the debugger. Winedbg shows 'Usage: blah blah'. Seems 'wine' calls it with incorrect parameters. - Wine fails to start any application, complaining about 'kthreads' something or the other.
I imagine that if I spend a few more weeks on the above problems, I can find patches in Wine CVS that fixes the exact problems I'm seeing.
I've already wasted a week of my time on problems that someone else already fixed. Spending two or more weeks doing this is not exactly on my fun-things-to-do-before-I-die list. I'd like to see a general solution to this entire problem.
What can be done to alleviate? ==============================
Once I find a patch which makes an earlier version actually compile, I'd like to propose that for a backport.
If approved/applied, the next person that checks out the release would check out a version containing the backported patches.
That would fix the problem and allow people to do proper regression testing, so developers and QA people can spend their time doing development and QA instead of redoing already done effort trying to get an older version to run.
I'm not sure how this would work with CVS. I'm from an SVN world where tagging is a 0-cost tree-copy operation, and any patches can be applied to the tag with history and all showing that the tag's been modified. If it can't work with CVS, perhaps the tags can be created in Troy's SVN repo and patches applied there? Not sure. Suggestions, please, people.