I hate to stir the pot, especially as an unknown in the community, but I've spent the last few hours reading WINE's history regarding DIB engines and it is pretty distressing.
I have seen expressions of frustration from many regarding the handling of the mostly-functional DIB engine that Massimo wrote. AJ's terseness on the issue is puzzling. I find it really weird that such a major and long-standing thing would just be left to languish without any detailed reasons. Though I do not believe it, you can start to see why the slashdotter described Codeweavers' corporate agenda as "evil"; it almost feels like Codeweavers is holding DIB hostage, just waiting for someone to get fed up and fork over the cash for it. This wouldn't be a problem of course if it appeared that Alexandre was willing to work with external developers on the DIB engine, but he has given very sparse criticism.
Should we all just accept that Massimo's engine is yet another DIB engine gone to waste and there is no future hope for it at all? Max's statements to that effect can be confusing out of context because of his non-native English, but everything I have seen indicates that he is totally willing to work on upstream merging if Alexandre is willing to cooperate. All cases of saying "it won't get merged" etc. by Max seem to reference AJ's reluctance to provide useful criticism on the code that gets posted.
It is not my intention to start another huge flamewar or thread on the matter. I just want to promote a reliable roadmap and help resolve this -- it seems like a total shame that there have been multiple DIB engines written and no parts of any of them were deemed worthy of inclusion in WINE. Max at least has indicated a desire to modify the code as necessary if a useful road map or ideation were provided, so why is it so hard to get the road map? Alexandre is right that the architecture is a lot of work, but I am not asking for him to write out a complete spec, and I don't think the community is, either; the main thing, as far as I can tell, is that the interaction and feedback on a major step forward for WINE has been minimal.
People are still reporting major speedups on bug #421 so I don't think this is a lost cause unworthy of effort as some suggest.
The most feedback I was able to find from Alexandre was on May 2009's DIB engine passing all tests thread at http://www.winehq.org/pipermail/wine-devel/2009-May/thread.html#75864 . Alexandre's single major standing complaints seem to be a lack of test cases and Massimo not establishing a good record with simple patches. Are those still valid reasons?
The complaint regarding driver loading was addressed, but it sounds like Alexandre wants it in the opposite direction (forwarding calls from winex11.drv (or whatever driver) to winedib.drv instead of the other way around). Is that a very serious problem?
So, if we can, let's get a few points down on what needs to be modified for DIB to make it into WINE, even as an experimental branch:
1) Write more gdi32 and any other relevant test cases to prove that the DIB engine is generally well-behaved 2) Reverse the DIB ordering and call winedib.drv only as needed, instead of passing everything through winedib.drv first. 3) ?
Or, in the case that Max's DIB is gone forever and is considered irreparable, why don't we discuss what's needed for a different DIB engine?
1) Write more gdi32 and any other relevant test cases to prove that the DIB engine is generally well-behaved 2) Works well with any wine display driver. (already in Max's code) 3) Applied and developed cooperatively and incrementally with the WINE community 4) Easily disabled / non-disruptive (already in Max's code) 5) ? What else needs to go here?
Is this accurate?
Others mentioned that benchmarks that prove the usefulness of a DIB engine might help. Do those still need to be performed?
Thanks for reading. From Jeff
On Sun, Aug 29, 2010 at 6:18 AM, Jeff Cook jeff@deserettechnology.com wrote:
Alexandre is right that the architecture is a lot of work, but I am not asking for him to write out a complete spec, and I don't think the community is, either; the main thing, as far as I can tell, is that the interaction and feedback on a major step forward for WINE has been minimal.
I know it seems frustrating of the lack of response. When I went through the summer of code, the most I got from Alexandre was, 'it doesn't work'. Well what doesn't work? And the honest answer from my own assessment was my engine wasn't finished--which is why I didn't push for a merge then. The school I went to required too much time for me to continue the work, so I put it on hold. That is when Max came in. He found that the code was useful for speeding up his AutoCad. And with the work he has done since then, others have found it useful. So it is not a waste.
The most feedback I was able to find from Alexandre was on May 2009's DIB engine passing all tests thread at http://www.winehq.org/pipermail/wine-devel/2009-May/thread.html#75864 . Alexandre's single major standing complaints seem to be a lack of test cases and Massimo not establishing a good record with simple patches. Are those still valid reasons?
Yes, very much so. The thread shows that there are problems exhibited by the engine which there are no test case for. So passing all tests that have been previously provided are not enough. That would be an area where simple patches can be done.
The dib engine 'blob' outside of the tree might be a maintenance issue for someone with the lack of time, and will be very hard to review. So something else will have to be done to get it mainline, if we ignore the implementation choice at the moment.
But certainly not all is lost. We now know more than we did before.
Jesse
Hi Jeff,
this has in fact come up before, and been addressed. The main problem with the current attempt, as far as I know, is that the risk of regressions is so high that only very high quality changes stand any chance of getting accepted. In the case of a DIB engine, the attempts have usually required accepting a large piece of new code wholesale, with a promise of future fixes. This is a very risky way to make a change, and Alexandre has repeatedly shown his preference for small, incremental changes wherever possible.
Without knowing the area in depth, I can only offer this commentary on Massimo's work: Massimo doesn't have a proven track record in the community, so I personally wouldn't trust his ability to minimize regressions, or to maintain his code until regressions he's caused have been fixed. This is nothing to do with him personally, I'd have the same expectations of any relatively new contributor.
As to the first, minimizing regressions, this is something a new developer can address by testing an area thoroughly. Often, the best path forward for new code is to introduce substantial new tests in the area before beginning to make functional changes.
I hate to stir the pot, especially as an unknown in the community, but I've spent the last few hours reading WINE's history regarding DIB engines and it is pretty distressing.
If you hate to do it, why do you do it? It's hard to maintain equanimity when you describe CodeWeavers as evil. Even if you do ascribe that to someone else, why on earth do you bother to repeat it? Such a sentiment is so absurd, it could best be accepted as a joke, but a bad one. Think about it: who has contributed the most, in real dollars, to Wine's development over time? Who releases their own Wine tree in source form? Who employs the most Wine contributors? Just because some pissed off slashdotter said something doesn't make it worthy of a second's thought.
Full disclosure: I once did a very small amount of contract work for CodeWeavers, approximately 6 years ago.
AJ's terseness on the issue is puzzling. I find it really weird that such a major and long-standing thing would just be left to languish without any detailed reasons. Though I do not believe it, you can start to see why the slashdotter described Codeweavers' corporate agenda as "evil";
Maybe to you, because you're new around here. When has Alexandre ever been long-winded? He's terse here just as he is elsewhere. If there is any single reason to reject a piece of code, that's enough. See e.g. Henri's comment on a patch the other day [1]: when we give negative feedback on a patch here, we usually don't point out every instance of an error, and expect the contributor to generalize from the feedback. Hence negative feedback here is usually terse.
it almost feels like Codeweavers is holding DIB hostage, just waiting for someone to get fed up and fork over the cash for it.
You can only hold hostage something that exists. There is no such beast as a working DIB engine.
. Alexandre's single major standing complaints seem to be a lack of test cases and Massimo not establishing a good record with simple patches. Are those still valid reasons?
Yes, of course they are. That's a valid reason for rejection of simple patches. Why shouldn't it be for major, high-risk-of-regression patches?
- Write more gdi32 and any other relevant test cases to prove that
the DIB engine is generally well-behaved
Yes, this is certainly needed.
Others mentioned that benchmarks that prove the usefulness of a DIB engine might help. Do those still need to be performed?
This could also help. If I recall correctly, Jeremy White mentioned at Wineconf 2008 that this was a major reason they haven't invested serious energy into one themselves: they had a hard time finding an application that they cared about that benefited significantly from a DIB engine. Usually, bottlenecks were elsewhere. Whether they didn't care about AutoCAD, or whether they hadn't tested with it, or whether my memory is just faulty, I'm not sure.
Hope that helps, --Juan
[1] http://www.winehq.org/pipermail/wine-devel/2010-August/086207.html
Juan Lang wrote:
Hi Jeff,
this has in fact come up before, and been addressed. The main problem with the current attempt, as far as I know, is that the risk of regressions is so high that only very high quality changes stand any chance of getting accepted. In the case of a DIB engine, the attempts have usually required accepting a large piece of new code wholesale, with a promise of future fixes. This is a very risky way to make a change, and Alexandre has repeatedly shown his preference for small, incremental changes wherever possible.
Juan and others:
One additional note: We should, as a project, not accept 'broken' code. I work with a real-time project and get paid for this that soon will have this policy in effect. No broken code, it is way to hard to go back and fix it later. This is one thing I like AJ for. The DIB engine is his one execption in that he wants it fully implemented and all in one piece.
Also, Max's code has shown up in another 'for pay' project and where implementation was done, works. Where implementation is not complete, it is seriously broken. The problem is where it works and does not work is not cleanly defined. Lots of 'surprises' and 'gotchas'.
Anyway, Jeff, please look at Max's code and then ask AJ for specific guidance on the how to work on this. This is a MAJOR part of the project that needs to be implemented. Don't feel insulted when you get reject after reject from AJ. As specific questions, and he does give answers. If he states "it's broken" he wants you to figure it out and the brokenness is very obvious. Yes, I've had patches rejected by him and that is all he said. I figured out the brokenness and fixed it. I've had to back off the patch as it used code that was rejected and now I know why.
James McKenzie
On Mon, Aug 30, 2010 at 8:00 AM, James McKenzie jjmckenzie51@earthlink.netwrote:
Also, Max's code has shown up in another 'for pay' project and where implementation was done, works. Where implementation is not complete, it is seriously broken. The problem is where it works and does not work is not cleanly defined. Lots of 'surprises' and 'gotchas'.
Hello,
Max's DIB Engine is built into Bordeaux : http://www.bordeauxgroup.com/mostly for nontechnical people who want to test Max's work in progress. The DIB Engine is turned off by default and unsupported apps can only be installed via the command line at this time.
As for CodeWeavers... They are some of the nicest people you will ever meet in your life, not to mention some of the brightest people on this planet. Jeremy did whoever offer to buy the "evil empire" last year for $1 down and the rest payed from future profits if memory serves me right. I'm not sure if Ballmer ever got back to them on the offer. :)
See : http://www.codeweavers.com/about/general/press/20090724/
I think my sig pretty much says everything else..
Cheers, Tom
On Sun, Aug 29, 2010 at 7:00 PM, James McKenzie jjmckenzie51@earthlink.net wrote:
One additional note: We should, as a project, not accept 'broken' code. I work with a real-time project and get paid for this that soon will have this policy in effect. No broken code, it is way to hard to go back and fix it later. This is one thing I like AJ for. The DIB engine is his one execption in that he wants it fully implemented and all in one piece.
Also, Max's code has shown up in another 'for pay' project and where implementation was done, works. Where implementation is not complete, it is seriously broken. The problem is where it works and does not work is not cleanly defined. Lots of 'surprises' and 'gotchas'.
Anyway, Jeff, please look at Max's code and then ask AJ for specific guidance on the how to work on this. This is a MAJOR part of the project that needs to be implemented. Don't feel insulted when you get reject after reject from AJ. As specific questions, and he does give answers. If he states "it's broken" he wants you to figure it out and the brokenness is very obvious. Yes, I've had patches rejected by him and that is all he said. I figured out the brokenness and fixed it. I've had to back off the patch as it used code that was rejected and now I know why.
James, I hate to be blunt, but you've been repeatedly told to stop giving advice while implying that you are a senior wine developer with lots of experience on the project. I've seen several (potential) new contributors take your advice, submit patches based on it, then have to rework a lot of it based on faulty advice.
Lots of people on wine-devel aren't wine developers, or contribute infrequently, but still give good advice, based on what they know and their experience. However, giving authoritative answers when you lack that authority/experience is damaging not only to those contributors, but to wine as a whole.
I think http://www.winehq.org/pipermail/wine-devel/2010-July/085495.html sums it up very nicely.
/rant
This could also help. If I recall correctly, Jeremy White mentioned at Wineconf 2008 that this was a major reason they haven't invested serious energy into one themselves: they had a hard time finding an application that they cared about that benefited significantly from a DIB engine. Usually, bottlenecks were elsewhere. Whether they didn't care about AutoCAD, or whether they hadn't tested with it, or whether my memory is just faulty, I'm not sure.
Yes, that's essentially right, although note that we did invest some fairly serious energy into the DIB engine prior to coming to that conclusion. (I remember how pleased Huw was that his benchmarks were 1000 times faster, and how displeased he was when it made Powerpoint slower...)
That effort all went into Wine, and I hope and believe that it has helped others as they have worked on the DIB engine.
But we don't have any secret stash of DIB engine code to further our evil plans for world domination. We rely on our gorgeous femme bots and Alexandre's magic 'all' patch for that <grin>.
Cheers,
Jeremy
On Sun, Aug 29, 2010 at 7:12 PM, Jeremy White jwhite@codeweavers.com wrote:
This could also help. If I recall correctly, Jeremy White mentioned at Wineconf 2008 that this was a major reason they haven't invested serious energy into one themselves: they had a hard time finding an application that they cared about that benefited significantly from a DIB engine. Usually, bottlenecks were elsewhere. Whether they didn't care about AutoCAD, or whether they hadn't tested with it, or whether my memory is just faulty, I'm not sure.
Yes, that's essentially right, although note that we did invest some fairly serious energy into the DIB engine prior to coming to that conclusion. (I remember how pleased Huw was that his benchmarks were 1000 times faster, and how displeased he was when it made Powerpoint slower...)
That effort all went into Wine, and I hope and believe that it has helped others as they have worked on the DIB engine.
But we don't have any secret stash of DIB engine code to further our evil plans for world domination. We rely on our gorgeous femme bots and Alexandre's magic 'all' patch for that <grin>.
Cheers,
Jeremy
To clarify, it's not about having secret DIB engine code, it's about saying "I guess we just won't find time to provide useful feedback until some company sponsors it...", as I saw several times during the old threads.
From Jeff
Jeff Cook wrote:
On Sun, Aug 29, 2010 at 7:12 PM, Jeremy White jwhite@codeweavers.com wrote:
This could also help. If I recall correctly, Jeremy White mentioned at Wineconf 2008 that this was a major reason they haven't invested serious energy into one themselves: they had a hard time finding an application that they cared about that benefited significantly from a DIB engine. Usually, bottlenecks were elsewhere. Whether they didn't care about AutoCAD, or whether they hadn't tested with it, or whether my memory is just faulty, I'm not sure.
Yes, that's essentially right, although note that we did invest some fairly serious energy into the DIB engine prior to coming to that conclusion. (I remember how pleased Huw was that his benchmarks were 1000 times faster, and how displeased he was when it made Powerpoint slower...)
That effort all went into Wine, and I hope and believe that it has helped others as they have worked on the DIB engine.
But we don't have any secret stash of DIB engine code to further our evil plans for world domination. We rely on our gorgeous femme bots and Alexandre's magic 'all' patch for that <grin>.
To clarify, it's not about having secret DIB engine code, it's about saying "I guess we just won't find time to provide useful feedback until some company sponsors it...", as I saw several times during the old threads.
This is a misunderstanding. Codeweavers figured out that they do not *need* the DIB engine aka it isn't the bottleneck. The estimates to get it implemented/merged where 6-12 man month of "architect level" Wine developer time. A huge time investment for little gain and thus Codeweavers decided to invest their resources in other areas that have a higher return of investment.
Now people keep asking about the DIB engine and how to motivate Codeweavers to work on that. I replied to that that in my *personal* opinion they won't do that unsexy and costly work unless somebody is waiving at them with some serious amount of money. That's the standard way in the enterprise world of rising the priority of a bug. But this is my *personal* opinion; I never ever have seen a Codeweavers employee say something like that. And I know Jeremy, he would *love* to get a DIB engine for free, especially as he knows its value in $$$ ;)
Now that "that waive with $$$" isn't very likely as the cost/benefit ratio isn't very good. More likely is that Codeweavers will start working on a Quartz driver and that work will make the DIB engine "cheaper" to implement. Again, this is my *personal* opinion.
bye michael
On Aug 29, 2010, at 8:49 PM, Jeff Cook wrote:
To clarify, it's not about having secret DIB engine code, it's about saying "I guess we just won't find time to provide useful feedback until some company sponsors it...", as I saw several times during the old threads.
What Jeremy neglected to say (although he's made this point before) is that Alexandre has, as a matter of contract, _complete_ independence as Wine maintainer. If, hypothetically, Jeremy wanted to and did "order" Alexandre to slow-walk the DIB engine (or anything else), Alexandre could and would simply ignore him. If Jeremy fired Alexandre for refusing such an order, Alexandre would have excellent grounds to sue CodeWeavers into the ground. Not that either of them is so inclined.
As far as Alexandre's famed terseness, you're completely mistaken if you imagine he's any less terse or more forthcoming with us CodeWeavers than he is with any other Wine contributor. In public or behind the scenes. It's just his nature. :)
Cheers, Ken
On Sun, Aug 29, 2010 at 2:18 PM, Jeff Cook jeff@deserettechnology.com wrote:
I hate to stir the pot, especially as an unknown in the community, but I've spent the last few hours reading WINE's history regarding DIB engines and it is pretty distressing.
I have seen expressions of frustration from many regarding the handling of the mostly-functional DIB engine that Massimo wrote. AJ's terseness on the issue is puzzling. I find it really weird that such a major and long-standing thing would just be left to languish without any detailed reasons. Though I do not believe it, you can start to see why the slashdotter described Codeweavers' corporate agenda as "evil"; it almost feels like Codeweavers is holding DIB hostage, just waiting for someone to get fed up and fork over the cash for it. This wouldn't be a problem of course if it appeared that Alexandre was willing to work with external developers on the DIB engine, but he has given very sparse criticism.
Should we all just accept that Massimo's engine is yet another DIB engine gone to waste and there is no future hope for it at all? Max's statements to that effect can be confusing out of context because of his non-native English, but everything I have seen indicates that he is totally willing to work on upstream merging if Alexandre is willing to cooperate. All cases of saying "it won't get merged" etc. by Max seem to reference AJ's reluctance to provide useful criticism on the code that gets posted.
It is not my intention to start another huge flamewar or thread on the matter. I just want to promote a reliable roadmap and help resolve this -- it seems like a total shame that there have been multiple DIB engines written and no parts of any of them were deemed worthy of inclusion in WINE. Max at least has indicated a desire to modify the code as necessary if a useful road map or ideation were provided, so why is it so hard to get the road map? Alexandre is right that the architecture is a lot of work, but I am not asking for him to write out a complete spec, and I don't think the community is, either; the main thing, as far as I can tell, is that the interaction and feedback on a major step forward for WINE has been minimal.
People are still reporting major speedups on bug #421 so I don't think this is a lost cause unworthy of effort as some suggest.
The most feedback I was able to find from Alexandre was on May 2009's DIB engine passing all tests thread at http://www.winehq.org/pipermail/wine-devel/2009-May/thread.html#75864 . Alexandre's single major standing complaints seem to be a lack of test cases and Massimo not establishing a good record with simple patches. Are those still valid reasons?
The complaint regarding driver loading was addressed, but it sounds like Alexandre wants it in the opposite direction (forwarding calls from winex11.drv (or whatever driver) to winedib.drv instead of the other way around). Is that a very serious problem?
So, if we can, let's get a few points down on what needs to be modified for DIB to make it into WINE, even as an experimental branch:
- Write more gdi32 and any other relevant test cases to prove that
the DIB engine is generally well-behaved 2) Reverse the DIB ordering and call winedib.drv only as needed, instead of passing everything through winedib.drv first. 3) ?
Or, in the case that Max's DIB is gone forever and is considered irreparable, why don't we discuss what's needed for a different DIB engine?
- Write more gdi32 and any other relevant test cases to prove that
the DIB engine is generally well-behaved 2) Works well with any wine display driver. (already in Max's code) 3) Applied and developed cooperatively and incrementally with the WINE community 4) Easily disabled / non-disruptive (already in Max's code) 5) ? What else needs to go here?
Is this accurate?
Others mentioned that benchmarks that prove the usefulness of a DIB engine might help. Do those still need to be performed?
Thanks for reading. From Jeff
Let me give my stance on the DIB engine but before I do that lets give a rough summary what the DIB engine is about because not everyone has a good idea what it is, some people think it is something magical.
A DIB(Section) is a type of surface you can draw to using GDI calls but you can also directly mess with its bits. On Windows DIB rendering is all done in software ('using the DIB engine'). In Wine we cheat by delegating most of the work to X11. This causes various challenges and some cause performance issues (though various of them are fixable even without a DIB engine).
Two of the challenges: - A copy of the DIB exists on the X-server and in system memory and both copies have to be synced. You don't always want to have them in sync (it might cause unneeded x11->sysmem copies). The syncing is done using memory protection tricks. If the system memory copy is not up to date, a segmentation fault is triggered on access to the bits and during that time the DIB is copied back from the X-server. This causes an issue for some bad applications which directly mess with the stack pointer while they are playing with the DIB.
- Depth conversion. When the DIB is at a different color depth (lets say it is 8-bit) than the X-server, the DIB is first converted to the depth of the X-server and afterwards it is converted back. Depending on the color depth this can be a heavy operation (various old games like Age of Empires are quite slow due to this). This used to be a big problem but due to XRender work for Wine, the conversion is not needed anymore for 16-bit/24-bit/32-bit DIBSections. It is still an issue for 8-bit and some other depths but for a big part it can be fixed (Alexandre also said it is more correct if a DIB is 8-bit, to just pass it as 8-bit to X).
Do we really need a DIB engine? For a part, 'time' has made the need for a DIB engine less important. Systems have become faster, X11 has become faster, XRender has helped and so on. There are applications which can really profit from a DIB engine but some of them can be made happy by improving the current code (e.g. don't perform color conversion and let X11 work at the depth of the DIB).
Personally I think the requirements for a DIB engine have changed and if we add one it should be part of a big Wine 2D rendering rewrite. In previous threads about a Wine quartz/cocoa driver, I mentioned that in Vista/Win7 Microsoft drastically changed their 2D engine and I think we should do something similar. The best way to summarize the changes is that their 'DIB engine' takes care of most tasks of the 'DDB engine'. Except for some common BitBlt ROPs, color filling, font rendering and a few other operations (this are the same calls Cairo/Cocoa/XRender supports) EVERYTHING is software rendered.
If we would go the Vista/Win7 way, the DIB engine would be something different from the previously proposed ones. We would move most of our rendering to the shared DIB/DDB engine. The few hw accelerated DDB operations could be implemented using OpenGL/XRender or it could even make sense to use Direct3D. The winex11 driver would implement these DDB options, would handle window management and input. Further it would be a lot easier to write a Cocoa driver for Mac.
I'm not sure if we will make such drastic changes at all. It would be a huge project (at least 1 year but likely more) of work.
Roderick
-
Austin English -
James McKenzie -
Jeff Cook -
Jeremy White -
Jesse Allen -
Juan Lang -
Ken Thomases -
Michael Stefaniuc -
Roderick Colenbrander -
Tom Wickline