Time-based releases idea

[Dan Kegel]

I just wrote up an idea related to release management for post-1.0
wine releases.  It's online at
  http://wiki.winehq.org/TimeBasedReleases
Essentially, the idea is to release in March and September,
in time for the April and October releases of Ubuntu.

For instance, following this strategy, we might plan to release
wine-1.2.0 in September 2008 or March 2009.

The alternative is to propose a set of criteria that wine-1.2
needs to meet, and working until they are met, which
is what we did for wine-0.9 and wine-1.0 (I think; it's kind
of hard to tell).

I look forward to discussing this idea... perhaps we shouldn't
bother to until after 1.0 is released, but I wanted to get it out
early so the discussion can begin in time for us to move on it
if we want to.
- Dan

[Austin English]

On Sun, May 4, 2008 at 10:13 PM, Dan Kegel <dank(a)kegel.com> wrote:
> I just wrote up an idea related to release management for post-1.0
>  wine releases.  It's online at
>   http://wiki.winehq.org/TimeBasedReleases
>  Essentially, the idea is to release in March and September,
>  in time for the April and October releases of Ubuntu.
>
> For instance, following this strategy, we might plan to release
>  wine-1.2.0 in September 2008 or March 2009.
>
> The alternative is to propose a set of criteria that wine-1.2
>  needs to meet, and working until they are met, which
>  is what we did for wine-0.9 and wine-1.0 (I think; it's kind
>  of hard to tell).
>
> I look forward to discussing this idea... perhaps we shouldn't
>  bother to until after 1.0 is released, but I wanted to get it out
>  early so the discussion can begin in time for us to move on it
>  if we want to.
>  - Dan

I don't think we should schedule our release schedule around Ubuntu's.
Just because it's very popular (and possibly our most widely used
target), doesn't mean we need to revolve around it. I'd say let's look
at the 1.2 buglist (along with the 1.0's we can't fix), set a goal to
get 1/2 of those fixed, test our 'supported' apps for regressions, and
release then.

Also might consider waiting until 1.0's hit for a month or so and see
what the biggest complaints are, and focus on fixing those for 1.2.

-Austin

[Steven Edwards]

On Sun, May 4, 2008 at 11:13 PM, Dan Kegel <dank(a)kegel.com> wrote:
> I look forward to discussing this idea... perhaps we shouldn't
>  bother to until after 1.0 is released, but I wanted to get it out
>  early so the discussion can begin in time for us to move on it
>  if we want to.

I think having semi-annual releases would be good but I don't think
they should be tied to any distro. Popularity of Linux distributions
shifts in the wind and with the ability for users to pull updates from
the internet, being tied to a release is largely irrelevant. Having
good criteria for semi-annual releases should be the topic at the next
wineconf though, unless that criteria falls to the benevolent dictator
in chief for life.

-- 
Steven Edwards

"There is one thing stronger than all the armies in the world, and
that is an idea whose time has come." - Victor Hugo

[Kai Blin]

On Monday 05 May 2008 05:13:16 Dan Kegel wrote:
> I just wrote up an idea related to release management for post-1.0
> wine releases.  It's online at
>   http://wiki.winehq.org/TimeBasedReleases
> Essentially, the idea is to release in March and September,
> in time for the April and October releases of Ubuntu.
>
> For instance, following this strategy, we might plan to release
> wine-1.2.0 in September 2008 or March 2009.

I'm against September, as it means we'll go into code freeze _again_ just 
before Summer of Code finishes.

While I agree that bi-annual releases sound like a good thing, I'm opposed to 
forcing it just to make Ubuntu's 8.10 schedule with 1.2.

Also, I'm not really sure what this'll mean in terms of features and bug 
fixes. We certainly don't want to keep people hanging without fixes for bugs 
for half a year.

We also don't want to slow down development more than needed.

Now, as you state on the wiki page, a bi-weekly release schedule is good for 
early adopters and developers, as it's centered on "Get my new cool feature 
in!" rather than on "Will this break Photoshop?". But, and I see you mention 
that yourself, "Will this break Photoshop?" is a really hard question to 
answer, especially for developers who don't own photoshop.

Bi-annual releases are good for people who already have their apps working, as 
prior to a release we probably should make sure nothing breaks. But as fixing 
new bugs sometimes is a bit hit-and-miss, only being able to try for the 
wider public once every six months isn't too great.

So assuming we manage to get wine test failures down to 0 during this code 
freeze, I'm happy to make sure none of my patches breaks anything there. If I 
can get a similar website for the apps we really don't want to regress, I'll 
of course try the same for that. But before we don't have that, I don't think 
it's a good idea.

Cheers,
Kai

-- 
Kai Blin
WorldForge developer  http://www.worldforge.org/
Wine developer        http://wiki.winehq.org/KaiBlin
Samba team member     http://www.samba.org/samba/team/
--
Will code for cotton.

[Reece Dunn]

2008/5/5 Kai Blin <kai.blin(a)gmail.com>:
> On Monday 05 May 2008 05:13:16 Dan Kegel wrote:
> > I just wrote up an idea related to release management for post-1.0
> > wine releases.  It's online at
> >   http://wiki.winehq.org/TimeBasedReleases
> > Essentially, the idea is to release in March and September,
> > in time for the April and October releases of Ubuntu.
> >
> > For instance, following this strategy, we might plan to release
> > wine-1.2.0 in September 2008 or March 2009.
>
> I'm against September, as it means we'll go into code freeze _again_ just
>  before Summer of Code finishes.

It might be better to align with SoC, so that we have a release when
SoC is finished (with time to properly absorb the changes) and another
before it starts, both roughly 6 months apart. This would then make it
easier for SoC participants not being in the middle of a code freeze.
> While I agree that bi-annual releases sound like a good thing, I'm opposed to
>  forcing it just to make Ubuntu's 8.10 schedule with 1.2.

I agree. If the Wine release schedule slips because of critical bugs,
or it is taking longer to apply a really cool SoC feature, then having
releases out-of-sync would increase the chance of getting that release
into Ubuntu. It also removes implied pressure that we have to release
on the specified dates. While the releases should not slip like Xorg
appears to be doing, likewise it should not ship if a critical app
does not work.
> Also, I'm not really sure what this'll mean in terms of features and bug
>  fixes. We certainly don't want to keep people hanging without fixes for bugs
>  for half a year.
>
> We also don't want to slow down development more than needed.
>
> Now, as you state on the wiki page, a bi-weekly release schedule is good for
>  early adopters and developers, as it's centered on "Get my new cool feature
>  in!" rather than on "Will this break Photoshop?". But, and I see you mention
>  that yourself, "Will this break Photoshop?" is a really hard question to
>  answer, especially for developers who don't own photoshop.

I like the bi-weekly releases. These can be seen as bleeding edge test
builds (and later, betas and release candidates). They are
developmental snapshots that allow people to try the bleeding edge
out, where things may break. I highly recommend that we keep these.
> Bi-annual releases are good for people who already have their apps working, as
>  prior to a release we probably should make sure nothing breaks. But as fixing
>  new bugs sometimes is a bit hit-and-miss, only being able to try for the
>  wider public once every six months isn't too great.

For people who don't have their apps working, who have a bug or bugs
that they are tracking, or who like to stay on the bleeding edge, we
should recommend that they try out the bi-weekly releases and continue
to file bug reports as is done currently.

For people who's app is working, we should recommend that they stick
to a stable release (unless they want to be on the bleeding edge, or
help out with testing).

Now, ideally, people should be moving with the bi-weekly releases to
pick up any regressions and to test bug fixes. Since Wine has fairly
aggressive development with the bi-weekly releases, it may be possible
to do quarterly stable releases.

The issue here is how to manage the stable releases. I would like to
see the aggressive bi-weekly development continue on wine.git. If we
have a wine-stable.git, or wine-1.x.y.git (like the Linux kernel),
that gets merged into with the bi-weekly release when code freeze
occurs, that can be stabilized to release. I don't know of a better
way that would not break either development model.
> So assuming we manage to get wine test failures down to 0 during this code
>  freeze, I'm happy to make sure none of my patches breaks anything there. If I
>  can get a similar website for the apps we really don't want to regress, I'll
>  of course try the same for that. But before we don't have that, I don't think
>  it's a good idea.

With the tests, it is critical that the failure rate is as low as possible.

I would also add that the wine todo results are factored in here as
well (not for this release, as the window for major changes has
closed). This is because those todo items will show up as bugs in real
applications, or highlight behavioural differences from Windows. This
is assuming that those tests are passing on Windows as well.

I would also argue that any bug must have a corresponding regression
test before being fixed, unless there is a good reason for there not
being one (e.g. it is a bug in winecfg). This way, Wine is less likely
to regress in the future, and the affected app is more likely to
continue to work after the fix.

- Reece

[Zachary Goldberg]

I think that a missing factor in making this decision is the shape of
an automatic test suite.  Its been mentioned a dozen times and has the
potential to tip the scales in favor of the time-based releases
(making QA easier -> shorter freezes).  In the event that we are able
to maintain QA (by test suite or other) then +1 for time based
releases.  Its very convenient for users to know when releases will be
and makes following software much easier as a whole.

For those who don't mind the cutting edge though, I don't see a
problem with still keeping minor revisions every other week with major
revisions every 6 months, e.g: 1.x.y, incrementing y every 2 weeks
incrementing x every 6 months

-Zach

[Maarten Lankhorst]

Hello Zachary,

2008/5/5 Zachary Goldberg <zgs(a)seas.upenn.edu>:
> I think that a missing factor in making this decision is the shape of
>  an automatic test suite.  Its been mentioned a dozen times and has the
>  potential to tip the scales in favor of the time-based releases
>  (making QA easier -> shorter freezes).  In the event that we are able
>  to maintain QA (by test suite or other) then +1 for time based
>  releases.  Its very convenient for users to know when releases will be
>  and makes following software much easier as a whole.
>
> For those who don't mind the cutting edge though, I don't see a
>  problem with still keeping minor revisions every other week with major
>  revisions every 6 months, e.g: 1.x.y, incrementing y every 2 weeks
>  incrementing x every 6 months
> I've been playing with automated tests, building tests on wine then
> running them on windows through cygwin+ interactive sshd service. But
> it seems to be flaky at best. A lot of tests still crash right now if
> I run them on my machine xp which sucks and shouldn't happen at all.
> Main culprits are the rpcrt4 and the crypt tests, but there might be
> others too (d3d9 does at the moment, might be a driver bug).

Cheers,
Maarten.

[Scott Ritchie]

Dan Kegel wrote:
> I just wrote up an idea related to release management for post-1.0
> wine releases.  It's online at
>   http://wiki.winehq.org/TimeBasedReleases
> Essentially, the idea is to release in March and September,
> in time for the April and October releases of Ubuntu.

You have my 120% support for this.  Although Ubuntu is certainly our
biggest distribution, I'd like to stress that by targeting March and
September, we're not just doing this for Ubuntu; we're doing it for
every distro that is timed to release shortly after the bi-annual Gnome
and X releases.

There is growing support for synchronizing distro release dates, not
only for the above reason, but also to make Linux as a whole look good.
> For instance, following this strategy, we might plan to release
> wine-1.2.0 in September 2008 or March 2009.
>
> The alternative is to propose a set of criteria that wine-1.2
> needs to meet, and working until they are met, which
> is what we did for wine-0.9 and wine-1.0 (I think; it's kind
> of hard to tell).

The alternative, truthfully, is choosing between shipping Ubuntu with a
2+months out of date Wine version or an untested one.  Either option sucks.

Let's face it, we effectively have time-based releases now, since with
the features-based model 1.0 kept getting pushed back for years and
years.  Now, that we've finally set a date, we're actually going to have
one ;)  I could be wrong, but I don't see any better dates to use than
March and September, just in time for the release candidates to hit the
betas of the upcoming distro releases.

Thanks,
Scott Ritchie

[Marcus Meissner]

On Mon, May 05, 2008 at 04:12:52AM -0700, Scott Ritchie wrote:
> Dan Kegel wrote:
> > I just wrote up an idea related to release management for post-1.0
> > wine releases.  It's online at
> >   http://wiki.winehq.org/TimeBasedReleases
> > Essentially, the idea is to release in March and September,
> > in time for the April and October releases of Ubuntu.
>
> You have my 120% support for this.  Although Ubuntu is certainly our
> biggest distribution, I'd like to stress that by targeting March and
> September, we're not just doing this for Ubuntu; we're doing it for
> every distro that is timed to release shortly after the bi-annual Gnome
> and X releases.

There aren't any except Ubuntu that do this I guess.

At least SUSE releases in an 8 month cycle currently.
> There is growing support for synchronizing distro release dates, not
> only for the above reason, but also to make Linux as a whole look good.

No, this is mostly Ubuntu wishful thinking as far as I see.
> The alternative, truthfully, is choosing between shipping Ubuntu with a
> 2+months out of date Wine version or an untested one.  Either option sucks.

I would really like to see what Alexandre thinks and how the post-1.0 
things work out.

Or if we have to put a release dude in, just like Dan now. ;)

Ciao, Marcus

[Alexandre Julliard]

Scott Ritchie <scott(a)open-vote.org> writes:
> The alternative, truthfully, is choosing between shipping Ubuntu with a
> 2+months out of date Wine version or an untested one.  Either option sucks.

I don't see how we can possibly have a tested release ready every time
some distro decides to ship. On the contrary, since distros don't give a
damn about Wine and usually do their best to break it (page zero issue
anyone?), we are better off releasing after a major distro release so
that we have a chance to find and fix the latest breakages first.
> Let's face it, we effectively have time-based releases now, since with
> the features-based model 1.0 kept getting pushed back for years and
> years.  Now, that we've finally set a date, we're actually going to have
> one ;)

It's still very much a feature-based model, only of course the desirable
features have been shifting as Microsoft shipped new stuff and people
wanted to run new apps before we supported the old ones properly...  A
deadline is of course necessary at some point, but the date was only set
once we got to the point that a release looked within reach.

Of course our next releases hopefully won't take 15 years each, but I
think it's too early to say if the next release will take 6 months or 2
years.

-- 
Alexandre Julliard
julliard(a)winehq.org

[Dan Kegel]

On Mon, May 5, 2008 at 5:12 AM, Alexandre Julliard <julliard(a)winehq.org> wrote:
> I don't see how we can possibly have a tested release ready every time
>  some distro decides to ship.

That wasn't the proposal.  The proposal was to ship every 6 months, and
to pick a release date that made some sense relative to any emerging
rhythm in distro releases.

FWIW, Fedora also seems to be on a six month release schedule;
http://fedoraproject.org/wiki/Releases/9/Schedule
says 8 released in November, and 9 will release in May.
> On the contrary, since distros don't give a
>  damn about Wine and usually do their best to break it (page zero issue
>  anyone?)

That wasn't the distro; that was an upstream kernel vulnerability fix
announced in February,
http://kerneltrap.org/Linux/Patching_CVE-2008-0600_Local_Root_Exploit
> we are better off releasing after a major distro release so
> that we have a chance to find and fix the latest breakages first.

If we want to catch breakages like the recent one early, we shouldn't
wait for the distros; we should run Wine with each new release of
the Linux kernel.
> It's still very much a feature-based model, only of course the desirable
>  features have been shifting as Microsoft shipped new stuff and people
>  wanted to run new apps before we supported the old ones properly...

You assert Wine's releases will be feature-based, but
I don't understand your reasoning yet.
- Dan

[Alexandre Julliard]

"Dan Kegel" <dank(a)kegel.com> writes:
> That wasn't the distro; that was an upstream kernel vulnerability fix
> announced in February,
> http://kerneltrap.org/Linux/Patching_CVE-2008-0600_Local_Root_Exploit

It's the distro that changed the mmap config, not the kernel. I'm not
sure I understand their reasoning, apparently this was an attempt to
work around the vulnerability without fixing the kernel. My point is
that if they cared about Wine they would have taken into account the
fact that it got broken, and hopefully found a different way of
addressing the issue.
> > we are better off releasing after a major distro release so
> > that we have a chance to find and fix the latest breakages first.
>
> If we want to catch breakages like the recent one early, we shouldn't
> wait for the distros; we should run Wine with each new release of
> the Linux kernel.

That doesn't help. Of all the breakages we've had over the years very
few were caused by the kernel; most were caused by distros shipping
unstable and untested packages, or broken default configs.
> > It's still very much a feature-based model, only of course the desirable
> >  features have been shifting as Microsoft shipped new stuff and people
> >  wanted to run new apps before we supported the old ones properly...
>
> You assert Wine's releases will be feature-based, but
> I don't understand your reasoning yet.

Making a stable release is a lot of work, particularly since given the
nature of Wine it's very hard to test it properly and make sure we are
not breaking things. We don't want to go through the process unless we
have some significant features to release, and significant features
can't be developed on a fixed time frame.

For instance, one feature we'd want in 1.2 is the DIB engine; nobody can
guarantee that it will be ready, tested, and debugged properly by
September. I'd much prefer to ship 1.2 with a working DIB engine say in
December, than with a broken one in September.

-- 
Alexandre Julliard
julliard(a)winehq.org

[Dan Kegel]

On Mon, May 5, 2008 at 6:54 AM, Alexandre Julliard <julliard(a)winehq.org> wrote:
> It's the distro that changed the mmap config, not the kernel. I'm not
>  sure I understand their reasoning, apparently this was an attempt to
>  work around the vulnerability without fixing the kernel.

Oh, right.  I think they were using belt and suspenders, on the theory that
more such bugs may be lurking.
> My point is that if they cared about Wine they would have taken
> into account the fact that it got broken, and hopefully found a different
> way of addressing the issue.

Indeed, they don't care about Wine enough yet.  I suspect that this will
change once enough of the latest Adobe apps run.
> Of all the breakages we've had over the years very
>  few were caused by the kernel; most were caused by distros shipping
>  unstable and untested packages, or broken default configs.

I stand corrected.
> Making a stable release is a lot of work, particularly since given the
>  nature of Wine it's very hard to test it properly and make sure we are
>  not breaking things.

The QA argument is a good one.  I would not feel comfortable doing
quick releases until we had a better automated app QA story -- even though
automated app QA never finds many bugs, it would provide some
insurance against brown-paper-bag regressions.  We would also need
a fairly large army of early adopters willing to try release candidates.
> We don't want to go through the process unless we
>  have some significant features to release, and significant features
>  can't be developed on a fixed time frame.
>  For instance, one feature we'd want in 1.2 is the DIB engine; nobody can
>  guarantee that it will be ready, tested, and debugged properly by
>  September. I'd much prefer to ship 1.2 with a working DIB engine say in
>  December, than with a broken one in September.

There are other features we might want in 1.2, for instance, Photoshop
CS3 support.
But I would happily ship a 1.2 that had none of these features
if it contained significant bugfixes, or if it had only CS3 support
and not a DIB Engine,
or vice versa.
- Dan

[Scott Ritchie]

Alexandre Julliard wrote:
> Scott Ritchie <scott(a)open-vote.org> writes:
> > The alternative, truthfully, is choosing between shipping Ubuntu with a
> > 2+months out of date Wine version or an untested one.  Either option sucks.
>
> I don't see how we can possibly have a tested release ready every time
> some distro decides to ship. On the contrary, since distros don't give a
> damn about Wine and usually do their best to break it (page zero issue
> anyone?), we are better off releasing after a major distro release so
> that we have a chance to find and fix the latest breakages first.

Why not target Wine around the beta release?  That way we still have
time to change and fix things that have to be done at the distro level.
 For instance, during the Ubuntu Hardy beta, I found a handful of bugs
that required changes in other packages than Wine, such as missing 32
bit versions of some of the libraries.

If we had committed Ubuntu to release with an older version of Wine that
didn't have those features, those bugs wouldn't have been found in time
for our dependencies to be fixed.  Users who managed to get the newer
release of Wine (such as from our website or with Ubuntu backports)
would have that feature permanently broken until the next Ubuntu
release, and in the meantime we'd likely get a bunch of bad bug reports.
> > Let's face it, we effectively have time-based releases now, since with
> > the features-based model 1.0 kept getting pushed back for years and
> > years.  Now, that we've finally set a date, we're actually going to have
> > one ;)
>
> It's still very much a feature-based model, only of course the desirable
> features have been shifting as Microsoft shipped new stuff and people
> wanted to run new apps before we supported the old ones properly...  A
> deadline is of course necessary at some point, but the date was only set
> once we got to the point that a release looked within reach.

Fair enough.  There really is nothing wrong with what happens in
practice: tons of ambitious new features are targeted for the next major
release, then after the date is set and begins to approach things get
more aggressively triaged.  You are, of course, free to delay a release
pending whatever feature you think is both doable and critical.
> Of course our next releases hopefully won't take 15 years each, but I
> think it's too early to say if the next release will take 6 months or 2
> years.

We don't need to make a new major release every 6 months for things to
time well, just some multiple of it.  The main benefit is the same as
having the biweekly releases - more users with a newer Wine.

I can't say for sure as I haven't worked on Wine C code, but I don't
think even an ambitious 6 month release cycle puts us in danger of
having a dramatic amount of lost work.  The move to GIT has likely
helped a lot, as its relatively easy to maintain branches for a feature
that won't make it into the next release. If the stable release is at
the wrong time, a developer could practically ignore the release
entirely and simply have his changes merged in later.


The biweekly release has served us very well, and it seems reasonable to
extend this sort of regularity to stable releases. Wine obviously
shouldn't recommend a particular distro or anything, but my experience
with Ubuntu and Wine has taught me that there are some very real and
tangible benefits to timed releases.  March and September seem like as
good months as any.

Thanks,
Scott Ritchie