Does wine send window messages without waiting apps to reenter userdll?

[Javier Pimás]

Hi! this is a very low-level technical question and my first one to this
list so please take with a grain of salt. I'm debugging a 32 bit app in
wine 1.6.2, on ubuntu 15.04 64 bits.

The app I'm debugging is a smalltalk vm, so it is very possessive with the
native stack, synchronization, callbacks and things like that. I'm having a
problem that happens in wine and not in windows, and the following thing I
describe might be the cause, it's very subtle.

The problem is related with non-queued messages, in a loop that calls
WaitMessage and does something more if there's a message. When running on
windows, WaitMessage blocks the thread until there's something (queued or
non-queued). If the message is non-queued, it is not immediately delivered
(adding frames to the stack by hand), but windows just awakes the thread
and waits it to reenter the OS (or userdll, I guess). I'm not sure if i can
trust wine to do the same, or if it awakes the process with a bunch of
frames pushed in the stack so that the non-queued message is delivered as
if WaitMessage never returned.

Here is a small idea of the subtle stack variations:

always in windows:

|        VM Callback        | <--- stack top
|      WndProc context      |
|           ....            |
| Next userdll call context | <- WaitMessage already returned
|         VM Callout        |
|           ....            |

------------------

never happens in windows (*i think*), possible in Wine?:

|        VM Callback        | <--- stack top
|      WndProc context      |
|           ....            |
|    WaitMessage context    | <--- never returned
|         VM Callout        |
|           ....            |

As I said, this is very low-level and subtle. Most apps wouldn't notice the
difference, but this is a VM. Is the second layout possible in wine?

Thanks for reading!

-- 
Javier Pimás
Ciudad de Buenos Aires

[Stefan Dösinger]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Am 2015-08-25 um 22:09 schrieb Javier Pimás:
> Hi! this is a very low-level technical question and my first one to
> this list so please take with a grain of salt. I'm debugging a 32
> bit app in wine 1.6.2, on ubuntu 15.04 64 bits.
> I recommend to switch to the development version of Wine if you're
> doing debugging like this.
> The app I'm debugging is a smalltalk vm, so it is very possessive
> with the native stack, synchronization, callbacks and things like
> that. I'm having a problem that happens in wine and not in windows,
> and the following thing I describe might be the cause, it's very
> subtle.
> I can't answer the core part of your question I'm afraid. I'm curious
> though why this would make a difference. Does the VM place something
> on the stack that the message handler expects to find? If so, how does
> it find it? Does it expect a certain stack depth as well?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBAgAGBQJV3bCaAAoJEN0/YqbEcdMwoXEP/Rss350C+9yjpzRoZKcXkhop
37QwmLrlkXg7Br0xWhgltfiIXpmRcrKq1EgGah0m9YTfSTjLbOIz80XszUELSCCc
lGnTOlkvETCJMv7EdinIVehngpm4phRugpfZSDdPNnx+orJn6T87PkbDhwTZsZ6u
g3TljQ1sLbF2O/AOKzr8S3CltIkf5oReHB29T4DnABbQqfXm0X5PhyKPevCtnWTG
od3PgQprwCve+DJmikZAVrqQtyms9xaryLh9Xc3XrYO8YNe2lxgfKYJY8NA3tDsk
nA+A1sNZferqIl5xUMooaquzU9471FS5pItXRPsnXJ5zwrzC47afgF47bnLpUmaQ
R2B6Vt8mUlB/KpNnhzzu9Q/RrEDwOj8PxmY/lhcSH2y4Y3VTAQCY2Mhrqc2tdJWT
li8R6OCF24pA1EM7q/+5gB2hedO4kMHcy5LcDvtxXbamIafxuUCM6loG5LR2VE+x
6qR1z+83CknyOOAm/MePNMDZqJ4dJmuM3V+8Dsv5YK5YtyM03i1Qy9l78jTkUGIG
2jVsRW1Ske+Uz50cBtago9z8S01BTMsLtctmQeFfmushfPW5BUjRLrJD257EngVH
lqNSBDSV/u41EiDjapN3ASkcWrEihdOyYQLJAfDjwu4vAN5MMO27B3N14wL2zKQ0
5+q0DrKfLGFUTgIQL0bR
=J5gb
-----END PGP SIGNATURE-----

[Javier Pimás]

Thanks for reading, sorry for the delay, I wanted to do more tests. Answer
between lines

On Wed, Aug 26, 2015 at 9:27 AM, Stefan Dösinger <stefandoesinger(a)gmail.com>
wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi,
>
> Am 2015-08-25 um 22:09 schrieb Javier Pimás:
> > Hi! this is a very low-level technical question and my first one to
> > this list so please take with a grain of salt. I'm debugging a 32
> > bit app in wine 1.6.2, on ubuntu 15.04 64 bits.
> > I recommend to switch to the development version of Wine if you're
> > doing debugging like this.
> > The app I'm debugging is a smalltalk vm, so it is very possessive
> > with the native stack, synchronization, callbacks and things like
> > that. I'm having a problem that happens in wine and not in windows,
> > and the following thing I describe might be the cause, it's very
> > subtle.
> > I can't answer the core part of your question I'm afraid. I'm curious
> > though why this would make a difference. Does the VM place something
> > on the stack that the message handler expects to find? If so, how does
> > it find it? Does it expect a certain stack depth as well?

It's not so much for the stack shape but that the VM expects executing app
code after returning from a WndProc. Before returning, the VM sets a
RETURNED_FROM_CALLBACK flag, that is checked on method activations (very
frequently). If WaitMessage doesn't return, a method activation doesn't
occur and the flag is left active. A second non-queued message then
reenters app code, and on the first method activation the flag is detected
the code isn't where it supposed (after returning) but in another callback,
confusing the app. This is not to mean the VM is bug free, the strange
thing is that it behaves differently in wine and windows.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQIcBAEBAgAGBQJV3bCaAAoJEN0/YqbEcdMwoXEP/Rss350C+9yjpzRoZKcXkhop
> 37QwmLrlkXg7Br0xWhgltfiIXpmRcrKq1EgGah0m9YTfSTjLbOIz80XszUELSCCc
> lGnTOlkvETCJMv7EdinIVehngpm4phRugpfZSDdPNnx+orJn6T87PkbDhwTZsZ6u
> g3TljQ1sLbF2O/AOKzr8S3CltIkf5oReHB29T4DnABbQqfXm0X5PhyKPevCtnWTG
> od3PgQprwCve+DJmikZAVrqQtyms9xaryLh9Xc3XrYO8YNe2lxgfKYJY8NA3tDsk
> nA+A1sNZferqIl5xUMooaquzU9471FS5pItXRPsnXJ5zwrzC47afgF47bnLpUmaQ
> R2B6Vt8mUlB/KpNnhzzu9Q/RrEDwOj8PxmY/lhcSH2y4Y3VTAQCY2Mhrqc2tdJWT
> li8R6OCF24pA1EM7q/+5gB2hedO4kMHcy5LcDvtxXbamIafxuUCM6loG5LR2VE+x
> 6qR1z+83CknyOOAm/MePNMDZqJ4dJmuM3V+8Dsv5YK5YtyM03i1Qy9l78jTkUGIG
> 2jVsRW1Ske+Uz50cBtago9z8S01BTMsLtctmQeFfmushfPW5BUjRLrJD257EngVH
> lqNSBDSV/u41EiDjapN3ASkcWrEihdOyYQLJAfDjwu4vAN5MMO27B3N14wL2zKQ0
> 5+q0DrKfLGFUTgIQL0bR
> =J5gb
> -----END PGP SIGNATURE-----

-- 
Javier Pimás
Ciudad de Buenos Aires

[Sebastian Lackner]

On 25.08.2015 22:09, Javier Pimás wrote:
> Hi! this is a very low-level technical question and my first one to this
> list so please take with a grain of salt. I'm debugging a 32 bit app in
> wine 1.6.2, on ubuntu 15.04 64 bits.

As Stefan Dösinger already suggested, upgrading to the latest development
version definitely wouldn't hurt. Maybe your issue was already fixed in the
meantime.
> The app I'm debugging is a smalltalk vm, so it is very possessive with the
> native stack, synchronization, callbacks and things like that. I'm having a
> problem that happens in wine and not in windows, and the following thing I
> describe might be the cause, it's very subtle.
>
> The problem is related with non-queued messages, in a loop that calls
> WaitMessage and does something more if there's a message. When running on
> windows, WaitMessage blocks the thread until there's something (queued or
> non-queued). If the message is non-queued, it is not immediately delivered
> (adding frames to the stack by hand), but windows just awakes the thread
> and waits it to reenter the OS (or userdll, I guess).

If you are concerned that WaitMessage immediately dispatches the messages,
Wine doesn't do that. Processing of window messages is only done in the
context of GetMessage and PeekMessage, no matter if its a "queued" or
"non-queued" message. In fact Wine doesn't really make a difference between
the two, it just assigns specific internal message numbers for non-queued
messages. To identify those the the highest bit (0x80000000) is set.
> I'm not sure if i can
> trust wine to do the same, or if it awakes the process with a bunch of
> frames pushed in the stack so that the non-queued message is delivered as
> if WaitMessage never returned.
>
> Here is a small idea of the subtle stack variations:
>
> always in windows:
>
> |        VM Callback        | <--- stack top
> |      WndProc context      |
> |           ....            |
> | Next userdll call context | <- WaitMessage already returned
> |         VM Callout        |
> |           ....            |
>
> ------------------
>
> never happens in windows (*i think*), possible in Wine?:
>
> |        VM Callback        | <--- stack top
> |      WndProc context      |
> |           ....            |
> |    WaitMessage context    | <--- never returned
> |         VM Callout        |
> |           ....            |
>
> As I said, this is very low-level and subtle. Most apps wouldn't notice the
> difference, but this is a VM. Is the second layout possible in wine?

That cannot happen in Wine.

However, that doesn't mean that handling of non-queued messages is already
implemented completely correct. Dmitry was working on this some time ago,
when you go back in the mailing list to April 2015 for example:
"user32: After handling an internal message give a chance to real message to arrive."

Regards,
Sebastian

[Sebastian Lackner]

On 26.08.2015 16:04, Sebastian Lackner wrote:
> If you are concerned that WaitMessage immediately dispatches the messages,
> Wine doesn't do that. Processing of window messages is only done in the
> context of GetMessage and PeekMessage, no matter if its a "queued" or
> "non-queued" message. In fact Wine doesn't really make a difference between
> the two, it just assigns specific internal message numbers for non-queued
> messages. To identify those the the highest bit (0x80000000) is set.

After reading the last part again I noticed that I wasn't really very precise,
because I somehow already assumed that you are talking about non-queued messages
caused by SetWindowPos and things like that. Those are implemented using internal
messages in Wine, and might be affected by the "bugs" mentioned in the last mail.

Besides that its of course also possible to send other non-queued messages
directly with SendMessage and similar functions. But in this case the answer would
be the same: WaitMessage does not dispatch them immediately, they are processed
during a call to PeekMessage or GetMessage.

Regards,
Sebastian

[Javier Pimás]

I wrote a small test which I attach here, very small program where you can
see if WndProc is called from WaitMessage. It fails, which I think shows
what I doubted is actually happening. Tested in the old 1.6.2 wine and also
in 1.7.50.

Cheers!

On Wed, Aug 26, 2015 at 11:29 AM, Sebastian Lackner <sebastian(a)fds-team.de>
wrote:
> On 26.08.2015 16:04, Sebastian Lackner wrote:
> > If you are concerned that WaitMessage immediately dispatches the
> > messages,
> > Wine doesn't do that. Processing of window messages is only done in the
> > context of GetMessage and PeekMessage, no matter if its a "queued" or
> > "non-queued" message. In fact Wine doesn't really make a difference
> > between
> > the two, it just assigns specific internal message numbers for non-queued
> > messages. To identify those the the highest bit (0x80000000) is set.
>
> After reading the last part again I noticed that I wasn't really very
> precise,
> because I somehow already assumed that you are talking about non-queued
> messages
> caused by SetWindowPos and things like that. Those are implemented using
> internal
> messages in Wine, and might be affected by the "bugs" mentioned in the
> last mail.
>
> Besides that its of course also possible to send other non-queued messages
> directly with SendMessage and similar functions. But in this case the
> answer would
> be the same: WaitMessage does not dispatch them immediately, they are
> processed
> during a call to PeekMessage or GetMessage.
>
> Regards,
> Sebastian

-- 
Javier Pimás
Ciudad de Buenos Aires

[Dmitry Timoshkov]

Javier Pimás <elpochodelagente(a)gmail.com> wrote:
> I wrote a small test which I attach here, very small program where you can
> see if WndProc is called from WaitMessage. It fails, which I think shows
> what I doubted is actually happening. Tested in the old 1.6.2 wine and also
> in 1.7.50.

It would be interesting to know what kind of messages you observe the test
app receives during WaitMessage().

-- 
Dmitry.

[Javier Pimás]

List of messages below. Just opened the window (which caused some) and
moved it through the screen (which caused others). These are

0x21    WM_MOUSEACTIVATE
0x46    WM_WINDOWPOSCHANGING
0x47    WM_WINDOWPOSCHANGED
0x03    WM_MOVE
0x1F    WM_CANCELMODE
0x86    WM_NCACTIVATE
0x06    WM_ACTIVATE
0x1C    WM_KILLFOCUS
0x08    WM_ACTIVATEAPP


In order:

# just for opening
Received message 21 while in WaitMessage
Received message 46 while in WaitMessage
Received message 47 while in WaitMessage
Received message 3 while in WaitMessage

# when moving
Received message 1f while in WaitMessage
Received message 46 while in WaitMessage
Received message 47 while in WaitMessage
Received message 3 while in WaitMessage
Received message 46 while in WaitMessage
Received message 47 while in WaitMessage
Received message 3 while in WaitMessage
Received message 46 while in WaitMessage
Received message 47 while in WaitMessage
Received message 3 while in WaitMessage
Received message 46 while in WaitMessage
Received message 47 while in WaitMessage
Received message 3 while in WaitMessage
Received message 46 while in WaitMessage
Received message 47 while in WaitMessage
Received message 3 while in WaitMessage
Received message 46 while in WaitMessage
Received message 47 while in WaitMessage
Received message 3 while in WaitMessage
Received message 46 while in WaitMessage
Received message 47 while in WaitMessage
Received message 3 while in WaitMessage
Received message 46 while in WaitMessage
Received message 47 while in WaitMessage
Received message 3 while in WaitMessage
Received message 46 while in WaitMessage
Received message 47 while in WaitMessage
Received message 3 while in WaitMessage
Received message 1f while in WaitMessage
Received message 86 while in WaitMessage
Received message 6 while in WaitMessage
Received message 1c while in WaitMessage
Received message 8 while in WaitMessage


On Fri, Aug 28, 2015 at 1:27 PM, Dmitry Timoshkov <dmitry(a)baikal.ru> wrote:
> Javier Pimás <elpochodelagente(a)gmail.com> wrote:
> > I wrote a small test which I attach here, very small program where you
> > can
> > see if WndProc is called from WaitMessage. It fails, which I think shows
> > what I doubted is actually happening. Tested in the old 1.6.2 wine and
> > also
> > in 1.7.50.
>
> It would be interesting to know what kind of messages you observe the test
> app receives during WaitMessage().
>
> --
> Dmitry.

-- 
Javier Pimás
Ciudad de Buenos Aires

[Sebastian Lackner]

Thanks for providing the testcase. It indeed looks like a wine bug. Would you
mind to open a bug report on bugs.winehq.org with your testcase and explanation?
Feel free to add me as CC.

The following quick & dirty hack fixes the issue (but will very likely break
other stuff): http://ix.io/kx0

Does it also help for your app?

Regards,
Sebastian

On 28.08.2015 17:52, Javier Pimás wrote:
> I wrote a small test which I attach here, very small program where you can
> see if WndProc is called from WaitMessage. It fails, which I think shows
> what I doubted is actually happening. Tested in the old 1.6.2 wine and also
> in 1.7.50.
>
> Cheers!
>
> On Wed, Aug 26, 2015 at 11:29 AM, Sebastian Lackner <sebastian(a)fds-team.de>
> wrote:
> > On 26.08.2015 16:04, Sebastian Lackner wrote:
> > > If you are concerned that WaitMessage immediately dispatches the
> > > messages,
> > > Wine doesn't do that. Processing of window messages is only done in the
> > > context of GetMessage and PeekMessage, no matter if its a "queued" or
> > > "non-queued" message. In fact Wine doesn't really make a difference
> > > between
> > > the two, it just assigns specific internal message numbers for non-queued
> > > messages. To identify those the the highest bit (0x80000000) is set.
> >
> > After reading the last part again I noticed that I wasn't really very
> > precise,
> > because I somehow already assumed that you are talking about non-queued
> > messages
> > caused by SetWindowPos and things like that. Those are implemented using
> > internal
> > messages in Wine, and might be affected by the "bugs" mentioned in the
> > last mail.
> >
> > Besides that its of course also possible to send other non-queued messages
> > directly with SendMessage and similar functions. But in this case the
> > answer would
> > be the same: WaitMessage does not dispatch them immediately, they are
> > processed
> > during a call to PeekMessage or GetMessage.
> >
> > Regards,
> > Sebastian

[Javier Pimás]

Ok, that should be 39182 <https://bugs.winehq.org/show_bug.cgi?id=39182>. I
downloaded wine source applied the patch and compiled (first time I ever do
that), but the output of the test didn't change.

On Sun, Aug 30, 2015 at 5:36 AM, Sebastian Lackner <sebastian(a)fds-team.de>
wrote:
> Thanks for providing the testcase. It indeed looks like a wine bug. Would
> you
> mind to open a bug report on bugs.winehq.org with your testcase and
> explanation?
> Feel free to add me as CC.
>
> The following quick & dirty hack fixes the issue (but will very likely
> break
> other stuff): http://ix.io/kx0
>
> Does it also help for your app?
>
> Regards,
> Sebastian
>
> On 28.08.2015 17:52, Javier Pimás wrote:
> > I wrote a small test which I attach here, very small program where you
> > can
> > see if WndProc is called from WaitMessage. It fails, which I think shows
> > what I doubted is actually happening. Tested in the old 1.6.2 wine and
> > also
> > in 1.7.50.
> >
> > Cheers!
> >
> > On Wed, Aug 26, 2015 at 11:29 AM, Sebastian Lackner <
> > sebastian(a)fds-team.de>
> > wrote:
> > > On 26.08.2015 16:04, Sebastian Lackner wrote:
> > > > If you are concerned that WaitMessage immediately dispatches the
> > > > messages,
> > > > Wine doesn't do that. Processing of window messages is only done in the
> > > > context of GetMessage and PeekMessage, no matter if its a "queued" or
> > > > "non-queued" message. In fact Wine doesn't really make a difference
> > > > between
> > > > the two, it just assigns specific internal message numbers for
> > > > non-queued
> > > > messages. To identify those the the highest bit (0x80000000) is set.
> > >
> > > After reading the last part again I noticed that I wasn't really very
> > > precise,
> > > because I somehow already assumed that you are talking about non-queued
> > > messages
> > > caused by SetWindowPos and things like that. Those are implemented using
> > > internal
> > > messages in Wine, and might be affected by the "bugs" mentioned in the
> > > last mail.
> > >
> > > Besides that its of course also possible to send other non-queued
> > > messages
> > > directly with SendMessage and similar functions. But in this case the
> > > answer would
> > > be the same: WaitMessage does not dispatch them immediately, they are
> > > processed
> > > during a call to PeekMessage or GetMessage.
> > >
> > > Regards,
> > > Sebastian

-- 
Javier Pimás
Ciudad de Buenos Aires

[Dmitry Timoshkov]

Sebastian Lackner <sebastian(a)fds-team.de> wrote:
> However, that doesn't mean that handling of non-queued messages is already
> implemented completely correct. Dmitry was working on this some time ago,
> when you go back in the mailing list to April 2015 for example:
> "user32: After handling an internal message give a chance to real message to arrive."

It would definitely help to see a test that reproduces the problem.

-- 
Dmitry.