Modifying Subsystem in Winelib executables
To fix one of the tests (shell32:shlexec) I'd like to copy the test executable and change the Subsystem type from IMAGE_SUBSYSTEM_WINDOWS_CUI (console app) to IMAGE_SUBSYSTEM_WINDOWS_GUI (GUI app) in the copy. Piece of cake for PE files, get the IMAGE_DOS_HEADER, follow it to the IMAGE_NT_HEADERS and just patch the OptionalHeaders.Subsystem field. My limited understanding of the structure of Winelib executables makes doing the same for testing on Wine quite a bit harder. I assume there's still a IMAGE_NT_HEADERS structure in there somewhere and with some research I'll probably be able to locate it in Linux ELF Winelib executables. What I'm wondering about is how big the can of worms I'm trying to open is? Is the encapsulation of PE headers in "native" executables the same for all OSes supported by Wine? Is changing a PE header field in (a copy of) a Winelib executable doomed to fail or is it a walk in the park? Ge.
Greg Geldorp <ggeldorp(a)vmware.com> writes:
My limited understanding of the structure of Winelib executables makes doing the same for testing on Wine quite a bit harder. I assume there's still a IMAGE_NT_HEADERS structure in there somewhere and with some research I'll probably be able to locate it in Linux ELF Winelib executables. What I'm wondering about is how big the can of worms I'm trying to open is? Is the encapsulation of PE headers in "native" executables the same for all OSes supported by Wine? Is changing a PE header field in (a copy of) a Winelib executable doomed to fail or is it a walk in the park?
It's doomed to fail, don't try that. If you need a specific format PE file generate one from scratch, don't try to modify an existing one. -- Alexandre Julliard julliard(a)winehq.org
On 09/29/2009 12:11 PM, Greg Geldorp wrote:
To fix one of the tests (shell32:shlexec) I'd like to copy the test executable and change the Subsystem type from IMAGE_SUBSYSTEM_WINDOWS_CUI (console app) to IMAGE_SUBSYSTEM_WINDOWS_GUI (GUI app) in the copy. Piece of cake for PE files, get the IMAGE_DOS_HEADER, follow it to the IMAGE_NT_HEADERS and just patch the OptionalHeaders.Subsystem field. My limited understanding of the structure of Winelib executables makes doing the same for testing on Wine quite a bit harder. I assume there's still a IMAGE_NT_HEADERS structure in there somewhere and with some research I'll probably be able to locate it in Linux ELF Winelib executables. What I'm wondering about is how big the can of worms I'm trying to open is? Is the encapsulation of PE headers in "native" executables the same for all OSes supported by Wine? Is changing a PE header field in (a copy of) a Winelib executable doomed to fail or is it a walk in the park?
Ge.
Can't we just compile shell32 tests as a GUI app? What other changes are needed to fix the errors on Vista+? (I know that we have popups now) -- Cheers, Paul.
From: Paul Vriens [mailto:paul.vriens.wine(a)gmail.com]
Can't we just compile shell32 tests as a GUI app?
I don't think that will work, but I'll look into it. Wine tests expect to have access to stdout. At the very least you won't see output when running the test manually on Windows. Not sure if it would affect running from winetest.exe.
What other changes are needed to fix the errors on Vista+? (I know that we have popups now)
Yeah, there are basically two problems with shell32:shlexec on Vista+. First is that there appears to be no way to suppress the popups that you get when there's no registered handler for a filename extension. I've got that one more or less under control, by detecting this condition and compensating for it when it first occurs and skipping similar tests afterwards. I can submit a patch for that right now if it helps. The second problem is when using DDE. My guess is the ShellExecuteEx() code for that looks something like this (at least, that's how I would write it :-)): try_establish_dde_connection(); if (no_connection) { CreateProcess(); WaitForInputIdle(); retry_establish_dde_connection(); } The child app is supposed to do something like this: WinMain() { setup_dde_server(); while (GetMessage()) { ... } } The WaitForInputIdle() in the parent should wait for the child to setup its DDE server, so when retry_establish_dde_conection() is called the DDE server is up and running. Except that this doesn't work for console apps. A console app can of course still have a message loop, but WaitForInputIdle() returns immediately for console apps. That means if the standard DDE setup is used you'll have a race condition when the child app is a console app, there's a good chance the childs setup_dde_server() won't have finished by the time the parent calls its retry_establish_dde_connection() function. For the shlexec test, the child launched can really only be shell32_test.exe, which hits this problem because that is a console app. There's a pretty cool hack in shlexec.c to work around this, I wish I would have thought of it myself :-). The idea is that the parent sets up the DDE server. That DDE server will decline the first attempt to establish a DDE connection. That will cause ShellExecuteEx() to launch the child process, a second copy of shell32_test.exe that will only check for correct parameters but will not try to do any DDE related stuff. In the mean time, we still have the DDE server in the parent running and when the parent calls retry_establish_dde_connection() its own DDE server responds and accepts the connection. The DDE server in the parent then checks the DDE conversation. Problem solved, except it doesn't work anymore on Vista+... At first I thought maybe ShellExecuteEx() will refuse to talk to a DDE server in its own process, but now I don't believe that's the problem. I do see the DDE server being probed for the initial connection which it refuses. It's just that no second connect request is received by the DDE server. Spying on messages sent by the parent makes me think that in retry_establish_dde_connection() it will only send WM_DDE_INITIATE messages to new windows (windows which were not present during the initial try_establish_dde_connection() call). That kind of makes sense, if a window didn't want to establish a conversation on a specific app/topic before it probably doesn't want to have that conversation at all, so there's no use in asking again. It breaks our hack though. I'm still trying to figure out a solution for this. Would be nice if the child could simply be a GUI app so WaitForInputIdle() would work as intended, but I don't see a way to make that happen. Another idea is to hook WaitForInputIdle and don't return until the child has set an event, indicating its DDE server is up and running. And obviously I'm open to other ideas. Ge.
On Tue, Sep 29, 2009 at 3:01 PM, Greg Geldorp <ggeldorp(a)vmware.com> wrote:
From: Paul Vriens [mailto:paul.vriens.wine(a)gmail.com]
Can't we just compile shell32 tests as a GUI app?
I don't think that will work, but I'll look into it. Wine tests expect to have access to stdout. At the very least you won't see output when running the test manually on Windows. Not sure if it would affect running from winetest.exe.
What other changes are needed to fix the errors on Vista+? (I know that we have popups now)
Yeah, there are basically two problems with shell32:shlexec on Vista+. First is that there appears to be no way to suppress the popups that you get when there's no registered handler for a filename extension. I've got that one more or less under control, by detecting this condition and compensating for it when it first occurs and skipping similar tests afterwards. I can submit a patch for that right now if it helps.
The second problem is when using DDE. My guess is the ShellExecuteEx() code for that looks something like this (at least, that's how I would write it :-)):
try_establish_dde_connection(); if (no_connection) { CreateProcess(); WaitForInputIdle(); retry_establish_dde_connection(); }
The child app is supposed to do something like this:
WinMain() { setup_dde_server(); while (GetMessage()) { ... } }
The WaitForInputIdle() in the parent should wait for the child to setup its DDE server, so when retry_establish_dde_conection() is called the DDE server is up and running. Except that this doesn't work for console apps. A console app can of course still have a message loop, but WaitForInputIdle() returns immediately for console apps. That means if the standard DDE setup is used you'll have a race condition when the child app is a console app, there's a good chance the childs setup_dde_server() won't have finished by the time the parent calls its retry_establish_dde_connection() function. For the shlexec test, the child launched can really only be shell32_test.exe, which hits this problem because that is a console app.
There's a pretty cool hack in shlexec.c to work around this, I wish I would have thought of it myself :-). The idea is that the parent sets up the DDE server. That DDE server will decline the first attempt to establish a DDE connection. That will cause ShellExecuteEx() to launch the child process, a second copy of shell32_test.exe that will only check for correct parameters but will not try to do any DDE related stuff. In the mean time, we still have the DDE server in the parent running and when the parent calls retry_establish_dde_connection() its own DDE server responds and accepts the connection. The DDE server in the parent then checks the DDE conversation. Problem solved, except it doesn't work anymore on Vista+...
At first I thought maybe ShellExecuteEx() will refuse to talk to a DDE server in its own process, but now I don't believe that's the problem. I do see the DDE server being probed for the initial connection which it refuses. It's just that no second connect request is received by the DDE server. Spying on messages sent by the parent makes me think that in retry_establish_dde_connection() it will only send WM_DDE_INITIATE messages to new windows (windows which were not present during the initial try_establish_dde_connection() call). That kind of makes sense, if a window didn't want to establish a conversation on a specific app/topic before it probably doesn't want to have that conversation at all, so there's no use in asking again. It breaks our hack though.
I'm still trying to figure out a solution for this. Would be nice if the child could simply be a GUI app so WaitForInputIdle() would work as intended, but I don't see a way to make that happen. Another idea is to hook WaitForInputIdle and don't return until the child has set an event, indicating its DDE server is up and running. And obviously I'm open to other ideas.
Can you not change the application type to GUI at runtime using MSVCRT's __set_app_type() function? Or does that do something different?
Ge.
Damjan Jovanovic
Hi Damjan,
From: Damjan Jovanovic
Can you not change the application type to GUI at runtime using MSVCRT's __set_app_type() function? Or does that do something different?
No, unfortunately that won't work for two reasons. First, it only affects how MSVCRT displays error messages (message box versus printing to stderr). Second, since it would be a runtime change there would still be a race condition. Thanks, Ge.
participants (4)
-
Alexandre Julliard -
Damjan Jovanovic -
Greg Geldorp -
Paul Vriens