December 2016 - Wine-Devel - list.winehq.org

Re: Testing Direct3D

by Fabian Maurer

On Saturday, November 26, 2016 12:42:16 AM CET you wrote: > Other options, once you have a test patch you could ask on > #winehackers or here on wine-devel for someone to give it a try. Still > suboptimal, yeah... > > I think there is a plan to add support for real boxes to the Testbot, > although that's probably not happening in the short term. I see, it would make sense for the 3D tests, at least. On Friday, November 25, 2016 9:36:23 PM CET Stefan Dösinger wrote: > Not really. You really want to have a real Windows machine if you want > to do d3d development. The testbot VMs do help with some things (e.g. > the d3d reference counting is implemented in Microsoft's part of the d3d > stack), but our tests are full with broken() statements that accept > behavior from VMs that are known to break games. > > If you need some one off test runs you can send me a patch and I can run > them on my Windows boxes for you. Thanks for the offer, I got a bit more storage and managed to set up a partition with win10 for dual boot, so I can test it myself, at least on my AMD card. Feel like this is a lot faster and less annoying for you and me.

Dec. 1, 2016

Re: [PATCH 2/2] winmm: Use a thread to send driver messages on Win9x (try 2)

by Andrew Eikum

On Thu, Dec 01, 2016 at 04:58:37PM -0200, Bruno Jesus wrote: > On Thu, Dec 1, 2016 at 4:44 PM, Andrew Eikum <aeikum(a)codeweavers.com> wrote: > > If I were to tackle this, I would reproduce the exact sequence of > > steps that the program takes, with identical parameters and structs, > > and see what happens when run on Windows. Then try to figure out which > > step or piece of data triggers the behavior switch. > > I'll see what I can do by following the relay logs. > Well, I meant specifically how it interacts with winmm. I think it's more likely that native winmm has some trigger to do the win9x threading behavior than it kills SuspendThread. But anything's possible. In any case, it might be best to wait until after the next release. It's easy to have unintended consequences in winmm. Andrew

Dec. 1, 2016

Re: [PATCH 2/2] winmm: Use a thread to send driver messages on Win9x (try 2)

by Bruno Jesus

On Thu, Dec 1, 2016 at 4:44 PM, Andrew Eikum <aeikum(a)codeweavers.com> wrote: >> I don't know what else to do, you can read some more at >> https://bugs.winehq.org/show_bug.cgi?id=3930#c18 >> > > Huh, interesting. Either Windows has magic for this game, or it uses > some other heuristic to determine whether to use the old behavior. > >> It would not be the first time Windows circumvents games problems. My >> patch is only touching Wine as configured for 9x, it brings no issues >> to the general user and makes Wine behave as 9x did. I cannot solve >> the mistery of how XP detects the game and avoids SuspendThread as my >> knowledge is limited and I believe it would require reverse >> engineering. >> > > To be clear, I'm not rejecting your idea. Whatever this heuristic is, > it would just replace the Windows version check in your patch so users > wouldn't have to switch the Windows version in winecfg. We still need > the separate thread implementation. Thanks, re-reading what I wrote may have sounded like I'm angry or something like that but it is just my simple English vocabulary. In a different point of view I could say that since the next stable is getting closer this solution would really affect the minimum amount of people possible, and that I don't have the skills to debug anything as my assembly knowledge is limited to mov, add, xor, cmp, jmp, jnz ;-) If we can bypass SuspendThread there wouldn't be a need for the thread, a hack that works for this bug is to comment out Suspend/ResumeThread in Wine code. But I guess it is harder than creating the thread since there is no simple way to do that in Windows API AFAICS. > If I were to tackle this, I would reproduce the exact sequence of > steps that the program takes, with identical parameters and structs, > and see what happens when run on Windows. Then try to figure out which > step or piece of data triggers the behavior switch. I'll see what I can do by following the relay logs. Best wishes, Bruno

Dec. 1, 2016

Re: [PATCH 2/2] winmm: Use a thread to send driver messages on Win9x (try 2)

by Andrew Eikum

On Thu, Dec 01, 2016 at 04:04:42PM -0200, Bruno Jesus wrote: > On Thu, Dec 1, 2016 at 3:29 PM, Andrew Eikum <aeikum(a)codeweavers.com> wrote: > > If the game works without compat mode, but your test doesn't, then > > there must be something else going on here. I wish we understood > > better why it works. > > I don't know what else to do, you can read some more at > https://bugs.winehq.org/show_bug.cgi?id=3930#c18 > Huh, interesting. Either Windows has magic for this game, or it uses some other heuristic to determine whether to use the old behavior. > It would not be the first time Windows circumvents games problems. My > patch is only touching Wine as configured for 9x, it brings no issues > to the general user and makes Wine behave as 9x did. I cannot solve > the mistery of how XP detects the game and avoids SuspendThread as my > knowledge is limited and I believe it would require reverse > engineering. > To be clear, I'm not rejecting your idea. Whatever this heuristic is, it would just replace the Windows version check in your patch so users wouldn't have to switch the Windows version in winecfg. We still need the separate thread implementation. If I were to tackle this, I would reproduce the exact sequence of steps that the program takes, with identical parameters and structs, and see what happens when run on Windows. Then try to figure out which step or piece of data triggers the behavior switch. Andrew

Dec. 1, 2016

Re: [PATCH 2/2] winmm: Use a thread to send driver messages on Win9x (try 2)

by Bruno Jesus

On Thu, Dec 1, 2016 at 3:29 PM, Andrew Eikum <aeikum(a)codeweavers.com> wrote: > On Thu, Dec 01, 2016 at 02:24:38PM -0200, Bruno Jesus wrote: >> No, manually setting compatibility is not required in XP (that is why >> IMO its is a shim, more below). Just the game installer needs >> compatibility or it complains about unsupported OS. >> >> I could not find any way to disable SuspendThread, I tried it back in >> [1] but got no replies, internet also did not help me at this subject. >> >> [1] https://www.winehq.org/pipermail/wine-devel/2015-November/110308.html >> >> To prove my point at [2] (source at [3]) there is a winmm_crosstest >> that calls SuspendThread inside the callback. If you run this at >> Windows >= XP it will get stuck. If you turn compatibility mode on it >> will pass as if nothing was called (tested on XP and 8). >> >> [2] http://alexa.pro.br/~bruno/wine/winmm_crosstest.exe >> [3] http://alexa.pro.br/~bruno/wine/winmm.txt >> > > If the game works without compat mode, but your test doesn't, then > there must be something else going on here. I wish we understood > better why it works. I don't know what else to do, you can read some more at https://bugs.winehq.org/show_bug.cgi?id=3930#c18 You can also read the changelog of the DLL file http://www.radgametools.com/msshist.htm, especifically this (which could or not be related to the problem, and the games were shipped with old versions.): --------- Changes from 3.03c to 3.03d (11-22-1995) Switched to a new thread synchronization technique which should fix deadlocking problems previously encountered by some 32-bit MSS applications. --------- It would not be the first time Windows circumvents games problems. My patch is only touching Wine as configured for 9x, it brings no issues to the general user and makes Wine behave as 9x did. I cannot solve the mistery of how XP detects the game and avoids SuspendThread as my knowledge is limited and I believe it would require reverse engineering. Best wishes, Bruno

Dec. 1, 2016

Re: [PATCH 2/2] winmm: Use a thread to send driver messages on Win9x (try 2)

by Andrew Eikum

On Thu, Dec 01, 2016 at 02:24:38PM -0200, Bruno Jesus wrote: > No, manually setting compatibility is not required in XP (that is why > IMO its is a shim, more below). Just the game installer needs > compatibility or it complains about unsupported OS. > > I could not find any way to disable SuspendThread, I tried it back in > [1] but got no replies, internet also did not help me at this subject. > > [1] https://www.winehq.org/pipermail/wine-devel/2015-November/110308.html > > To prove my point at [2] (source at [3]) there is a winmm_crosstest > that calls SuspendThread inside the callback. If you run this at > Windows >= XP it will get stuck. If you turn compatibility mode on it > will pass as if nothing was called (tested on XP and 8). > > [2] http://alexa.pro.br/~bruno/wine/winmm_crosstest.exe > [3] http://alexa.pro.br/~bruno/wine/winmm.txt > If the game works without compat mode, but your test doesn't, then there must be something else going on here. I wish we understood better why it works. Andrew

Dec. 1, 2016

Re: [PATCH 2/2] winmm: Use a thread to send driver messages on Win9x (try 2)

by Bruno Jesus

On Thu, Dec 1, 2016 at 1:14 PM, Andrew Eikum <aeikum(a)codeweavers.com> wrote: > On Thu, Dec 01, 2016 at 12:27:43PM -0200, Bruno Jesus wrote: >> On Thu, Dec 1, 2016 at 12:11 PM, Andrew Eikum <aeikum(a)codeweavers.com> wrote: >> > This seems like something you should be able to write a test for. I'm >> > also curious how this manages to work on Windows >= XP >> >> The test is at [1], if you run in Win9x it will fail (Win9x uses >> different thread), while running at >= XP it will succeed (>=XP uses >> same thread). This works in XP due to a shim that blindly ignores the >> SuspendThread/ResumeThread calls when inside a WinMM callback. >> >> [1] http://source.winehq.org/source/dlls/winmm/tests/wave.c#0570 >> > > See, I told you we could write a test for it. I already did ;) Sorry for not saying before that the test previously existed. > I'm still not sure exactly how this shim works. Is that something we > can replicate in Wine instead of this threaded solution? Requiring > users to switch the Windows version isn't ideal. Does this game > require you to set 9x compatibility mode on Windows? No, manually setting compatibility is not required in XP (that is why IMO its is a shim, more below). Just the game installer needs compatibility or it complains about unsupported OS. I could not find any way to disable SuspendThread, I tried it back in [1] but got no replies, internet also did not help me at this subject. [1] https://www.winehq.org/pipermail/wine-devel/2015-November/110308.html To prove my point at [2] (source at [3]) there is a winmm_crosstest that calls SuspendThread inside the callback. If you run this at Windows >= XP it will get stuck. If you turn compatibility mode on it will pass as if nothing was called (tested on XP and 8). [2] http://alexa.pro.br/~bruno/wine/winmm_crosstest.exe [3] http://alexa.pro.br/~bruno/wine/winmm.txt >> > You need to be more careful about thread shutdown. See >> > WINMM_StartDevicesThread and WINMM_DevicesThreadProc. I think you >> > could use the same logic, including WINMM_DevicesThreadDone, to >> > determine when to quit your new thread. >> >> I tried understand what you say but I'm not sure yet. >> WINMM_DeleteWaveform is only called on process detach, can the DLL >> live after that? >> > > The issue is that the callback thread may still be running after you > exit DllMain and the DLL is unloaded on a different thread, which will > cause a crash. Makes sense to me. > Because you can't use the Wait functions during DllMain, you need to > ensure no winmm code is being run on any thread *before* > PROCESS_DETACH is called. This means your thread needs to hold a > reference to the module to ensure it isn't unloaded. However, some > apps require winmm to be able to unload, so the thread can't just hold > the reference forever. Since your callback thread will only do useful > work if devices are opened (right?), you can use > WINMM_DevicesThreadDone to determine when to quit without depending on > Wait in DllMain. Finally, FreeLibraryAndExitThread allows you to exit > winmm code permanently in order to free the reference safely, in case > your thread is the last reference holder. Yes, the thread is only useful when devices are opened. > Note that this all means you'll need to start the callback thread > dynamically instead of just once. Since your thread and the existing > devices thread have the same lifespans (right?), I think you can just > piggyback on WINMM_StartDevicesThread and start both threads in there. You are right, the lifespans are the same. > Also, double-check that g_devthread_token is incremented in all > instances where your callback thread may be used while no devices > exist. This situation might never occur, but double-check to make > sure. > > I thought we had a bug about this, but I can't find it. See commit > 2d76befbddc798f02812fe48a12ab2b80d25181b. > > Hope that makes sense, it took a while to wrap my head around it when > I wrote that commit... Thanks, it all makes sense. I have to study further now. > Andrew Thank you very much again.

Dec. 1, 2016

Re: [PATCH 2/2] winmm: Use a thread to send driver messages on Win9x (try 2)

by Andrew Eikum

On Thu, Dec 01, 2016 at 12:27:43PM -0200, Bruno Jesus wrote: > On Thu, Dec 1, 2016 at 12:11 PM, Andrew Eikum <aeikum(a)codeweavers.com> wrote: > > This seems like something you should be able to write a test for. I'm > > also curious how this manages to work on Windows >= XP > > The test is at [1], if you run in Win9x it will fail (Win9x uses > different thread), while running at >= XP it will succeed (>=XP uses > same thread). This works in XP due to a shim that blindly ignores the > SuspendThread/ResumeThread calls when inside a WinMM callback. > > [1] http://source.winehq.org/source/dlls/winmm/tests/wave.c#0570 > See, I told you we could write a test for it. I already did ;) I'm still not sure exactly how this shim works. Is that something we can replicate in Wine instead of this threaded solution? Requiring users to switch the Windows version isn't ideal. Does this game require you to set 9x compatibility mode on Windows? > > You need to be more careful about thread shutdown. See > > WINMM_StartDevicesThread and WINMM_DevicesThreadProc. I think you > > could use the same logic, including WINMM_DevicesThreadDone, to > > determine when to quit your new thread. > > I tried understand what you say but I'm not sure yet. > WINMM_DeleteWaveform is only called on process detach, can the DLL > live after that? > The issue is that the callback thread may still be running after you exit DllMain and the DLL is unloaded on a different thread, which will cause a crash. Because you can't use the Wait functions during DllMain, you need to ensure no winmm code is being run on any thread *before* PROCESS_DETACH is called. This means your thread needs to hold a reference to the module to ensure it isn't unloaded. However, some apps require winmm to be able to unload, so the thread can't just hold the reference forever. Since your callback thread will only do useful work if devices are opened (right?), you can use WINMM_DevicesThreadDone to determine when to quit without depending on Wait in DllMain. Finally, FreeLibraryAndExitThread allows you to exit winmm code permanently in order to free the reference safely, in case your thread is the last reference holder. Note that this all means you'll need to start the callback thread dynamically instead of just once. Since your thread and the existing devices thread have the same lifespans (right?), I think you can just piggyback on WINMM_StartDevicesThread and start both threads in there. Also, double-check that g_devthread_token is incremented in all instances where your callback thread may be used while no devices exist. This situation might never occur, but double-check to make sure. I thought we had a bug about this, but I can't find it. See commit 2d76befbddc798f02812fe48a12ab2b80d25181b. Hope that makes sense, it took a while to wrap my head around it when I wrote that commit... Andrew

Dec. 1, 2016

Re: [PATCH 1/2] server: Store async list in process object and use that to find async in cansel_async request.

by Alexandre Julliard

Jacek Caban <jacek(a)codeweavers.com> writes: > On 01.12.2016 12:52, Alexandre Julliard wrote: >> Jacek Caban <jacek(a)codeweavers.com> writes: >> >>>> We'd probably need to clear them explicitly at process exit. >>> I agree that we should. I will send a new series implementing that. I >>> tested that with the attached patch (I can't send it to wine-patches yet >>> because it uses pipes in message mode). >> Thread exit would be interesting to test too. > > I tested that and it seems that Windows aborts asyncs on thread exit. > It's easy to implement aborting itself in Wine, but I must admit that I > have very mixed feelings about it. > > I would expect to be able to initialize an async I/O in one thread and > handle its completion in another. That's how threadpool - based > applications are likely to work and, maybe not literally, but [1] gives > an impression that it's supported. I tried to test threadpool with the > attached test, but it throws an exception in TerminateThread call, not > allowing me to terminate that thread. That makes me think that it > supports splitting async I/O between threads by heavily ensuring that > affected threads are not terminated. We don't have something like that > in Wine and I find it very risky to implement async cancellation on > thread exit at this point. It looks like it would need more work, yes. I guess for now, freeing them at process exit is still better than not freeing them at all ;-) Thanks for checking this! -- Alexandre Julliard julliard(a)winehq.org

Dec. 1, 2016

Re: [PATCH] iphlpapi: Ensure GetIpAddrTable returns loopback addresses in the end

by Alexandre Julliard

Bruno Jesus <00cpxxx(a)gmail.com> writes: > On Thu, Dec 1, 2016 at 11:43 AM, Alexandre Julliard <julliard(a)winehq.org> wrote: >>> --- >>> dlls/iphlpapi/iphlpapi_main.c | 27 +++++++++++++++++++++------ >>> dlls/iphlpapi/tests/iphlpapi.c | 4 ++++ >>> 2 files changed, 25 insertions(+), 6 deletions(-) >> >> This breaks the tests here: >> >> ../../../tools/runtest -q -P wine -T ../../.. -M ws2_32.dll -p ws2_32_test.exe.so sock && touch sock.ok >> sock.c:2489: Tests skipped: SOCK_RAW is not supported >> sock.c:6947: Test failed: got 11002 expected WSAHOST_NOT_FOUND >> sock.c:6948: Test failed: expected 11001, got 11002 >> sock.c:7197: Test failed: got 11002 expected WSAHOST_NOT_FOUND >> sock.c:7198: Test failed: expected 11001, got 11002 >> sock.c:8648: Tests skipped: Cannot test SIO_ADDRESS_LIST_CHANGE, interactive tests must be enabled >> Makefile:208: recipe for target 'sock.ok' failed >> make: *** [sock.ok] Error 4 > > Looks like an internet hiccup, do you mind testing again? I usually > test all network related DLL's when changing network code I here I > have no problems. Specially because what you got is TRY_AGAIN error. It does seem random, yes, sorry about that. I don't remember ever seeing that error before your change, but maybe that was just luck. -- Alexandre Julliard julliard(a)winehq.org

Dec. 1, 2016