Hello all,
I was working on a test for MSI type 1 custom actions, i.e. functions exported from a given DLL and called by the install script. I've tested both with an independent file and in the Wine test environment, and I've found that I can successfully export functions from the executable itself with DECLSPEC_EXPORT, and MSI will successfully execute the function, as the attached patch demonstrates.
The function returns ERROR_SUCCESS, and changing this to other values gets reflected in the install's error code appropriately. However, as soon as I add a function call to the custom action, the install fails with ERROR_INSTALL_FAILURE, and the function is never called—making it look, in all probability, like a page fault occurred. This is the case when compiling with mingw-w64, both inside the Wine test environment and in an independent file.
Notably I have no trouble calling DLLs taken from real MSI packages in this way. I also have no trouble calling the function if I use GetModuleHandle() and GetProcAddress() directly, nor do I run into any trouble whatsoever running the custom action on Linux. It is specifically this environment—running this function as part of a custom action, on Windows.
Does anyone know, or have a hunch, as to what the problem might be here? For what it's worth, it's known that native MSI uses a separate process to execute custom actions, although I don't know how this would lend any clue to what's going wrong.
--Zeb
On Thu, 2017-10-12 at 23:09 -0500, Zebediah Figura wrote:
I was working on a test for MSI type 1 custom actions, i.e. functions exported from a given DLL and called by the install script. I've tested both with an independent file and in the Wine test environment, and I've found that I can successfully export functions from the executable itself with DECLSPEC_EXPORT, and MSI will successfully execute the function, as the attached patch demonstrates.
The function returns ERROR_SUCCESS, and changing this to other values gets reflected in the install's error code appropriately. However, as soon as I add a function call to the custom action, the install fails with ERROR_INSTALL_FAILURE, and the function is never called—making it look, in all probability, like a page fault occurred. This is the case when compiling with mingw-w64, both inside the Wine test environment and in an independent file.
Notably I have no trouble calling DLLs taken from real MSI packages in this way. I also have no trouble calling the function if I use GetModuleHandle() and GetProcAddress() directly, nor do I run into any trouble whatsoever running the custom action on Linux. It is specifically this environment—running this function as part of a custom action, on Windows.
Does it fail on any function call or just calls back into the test execu table? Can you call a kernel API? Perhaps it's some sort of protection scheme.
On 10/13/2017 02:16 AM, Hans Leidekker wrote:
On Thu, 2017-10-12 at 23:09 -0500, Zebediah Figura wrote:
I was working on a test for MSI type 1 custom actions, i.e. functions exported from a given DLL and called by the install script. I've tested both with an independent file and in the Wine test environment, and I've found that I can successfully export functions from the executable itself with DECLSPEC_EXPORT, and MSI will successfully execute the function, as the attached patch demonstrates.
The function returns ERROR_SUCCESS, and changing this to other values gets reflected in the install's error code appropriately. However, as soon as I add a function call to the custom action, the install fails with ERROR_INSTALL_FAILURE, and the function is never called—making it look, in all probability, like a page fault occurred. This is the case when compiling with mingw-w64, both inside the Wine test environment and in an independent file.
Notably I have no trouble calling DLLs taken from real MSI packages in this way. I also have no trouble calling the function if I use GetModuleHandle() and GetProcAddress() directly, nor do I run into any trouble whatsoever running the custom action on Linux. It is specifically this environment—running this function as part of a custom action, on Windows.
Does it fail on any function call or just calls back into the test execu table? Can you call a kernel API? Perhaps it's some sort of protection scheme.
It fails on any call into the API, so I have no way of checking if it's failing on internal function calls.
Zebediah Figura z.figura12@gmail.com writes:
Does anyone know, or have a hunch, as to what the problem might be here? For what it's worth, it's known that native MSI uses a separate process to execute custom actions, although I don't know how this would lend any clue to what's going wrong.
In a separate process you'd already have the MSI server exe running, so you can't load a second exe. What probably happens is that the exe module is loaded but its imports are not resolved.
On 10/13/2017 02:21 AM, Alexandre Julliard wrote:
Zebediah Figura z.figura12@gmail.com writes:
Does anyone know, or have a hunch, as to what the problem might be here? For what it's worth, it's known that native MSI uses a separate process to execute custom actions, although I don't know how this would lend any clue to what's going wrong.
In a separate process you'd already have the MSI server exe running, so you can't load a second exe. What probably happens is that the exe module is loaded but its imports are not resolved.
Why does this happen for the test module but not for existing custom action DLLs?
Zebediah Figura z.figura12@gmail.com writes:
On 10/13/2017 02:21 AM, Alexandre Julliard wrote:
Zebediah Figura z.figura12@gmail.com writes:
Does anyone know, or have a hunch, as to what the problem might be here? For what it's worth, it's known that native MSI uses a separate process to execute custom actions, although I don't know how this would lend any clue to what's going wrong.
In a separate process you'd already have the MSI server exe running, so you can't load a second exe. What probably happens is that the exe module is loaded but its imports are not resolved.
Why does this happen for the test module but not for existing custom action DLLs?
DLLs can be loaded in the server process, but an exe can't.