The DefDlgProc16, DefDlgProcA and DefDlgProcW call CallWindowProc16, CallWindowProcA and CallWindowProcW to pass messages to the dialog procedures. The problem with this is that a dialog procedure is not a window procedure, despite having a similar signature. The dialog procedures return a BOOL rather than an LRESULT - the value that would be returned by a window procedure is returned by 'SetWindowLongPtrW( hwnd, DWLP_MSGRESULT, ... );' for a dialog procedure, and the return value of the dialog procedure is a BOOL indicating whether the message was processed.
This causes problems for messages like WM_GETTEXT, where the return value of the message is needed for (and modified by) post-processing when calling across a W->A or A->W boundary.
It seemed that the tidyest fix would be to create three new routines internal to dlls/user: "call_dialog_proc_a", "call_dialog_proc_w" and "call_dialog_proc16". These three routines would be substantially the same as CallWindowProcA, CallWindowProcW and CallWindowProc16, except that they would pass a flag to the 6 cross-call case handling routines to let the routine know it is calling a dialog procedure and needs to use the value for DWLP_MSGRESULT for its post-processing.
The problem with this is that two of the cross-call handling routines (the 16->W and 16->A ones) are exported:
__wine_call_wndproc_32A __wine_call_wndproc_32W
They do not appear to be used anywhere outside of the file they are defined in, and it seems unlikely an app could ever have a legitimate use for them. Vitaliy suggested maybe they may have been inadvertently missed in a cleanup of cross-DLL calls.
So, what I propose to do is:
1. make __wine_call_wndproc_32A and __wine_call_wndproc_32W static (and hence not exported)
2. Add a parameter to each of __wine_call_wndproc_32A, __wine_call_wndproc_32W, WINPROC_CallProc32WTo32A, WINPROC_CallProc32WTo16, WINPROC_CallProc32ATo16 and WINPROC_CallProc32ATo32W to indicate that the cross-call routine should get the LRESULT from (and set it into) the DWLP_MSGRESULT value.
3. Rename the existing CallWindowProc* routines to (static) call_window_proc*, add the flag parameter there as well, and create new CallWindowProc* and call_dialog_proc_* routines to call the renamed routines with the appropriate flag.
Any objections?