[Bug 51378] New: Failures with `DetourCreateProcessWithDllEx` for Microsoft's Detours Library

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=51378

Bug ID: 51378
           Summary: Failures with `DetourCreateProcessWithDllEx` for
                    Microsoft's Detours Library
           Product: Wine
           Version: 6.11
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: -unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

Created attachment 70234
  --> https://bugs.winehq.org/attachment.cgi?id=70234
stderr from WINEDEBUG

I am trying to work with a third-party tool that use Microsoft's Detours
Library:

https://github.com/microsoft/Detours

but it seems that the API call `DetourCreateProcessWithDllEx` inside of Detours
does not work correctly when run under Wine.

The implementation for `DetourCreateProcessWithDllEx{A/W}` can be found here:

https://github.com/microsoft/Detours/blob/bc7688373bbbb185277788c04f68ae67b0...

https://github.com/microsoft/Detours/blob/bc7688373bbbb185277788c04f68ae67b0...

Helpfully, Detours includes with a sample application `withdll.exe` that
utilises the `DetourCreateProcessWithDllEx` API:

https://github.com/microsoft/Detours/blob/fe7216c037c898b1f65330eda85e6146b6...

I built Detours from git (fe7216c037c898b1f65330eda85e6146b6e3cb85) on Windows
10 (19042.1052) using MSVC 19.29.30038.1; this gave me the resulting binaries
`withdll.exe` and `simple64.dll`. Running `withdll.exe` to inject
`simple64.dll` on Windows 10 gives:

```
C:\Users\avj\Downloads\Detours-master\bin.X64>withdll.exe /d:simple64.dll
withdll.exe
withdll.exe: Starting: `withdll.exe'
withdll.exe:   with
`C:\Users\avj\Downloads\Detours-master\bin.X64\simple64.dll'
simple64.dll: Starting.
simple64.dll: Detoured SleepEx().
Usage:
    withdll.exe [options] [command line]
Options:
    /d:file.dll   : Start the process with file.dll.
    /v            : Verbose, display memory at start.
    /?            : This help screen.
simple64.dll: Removed SleepEx() (result=0), slept 0 ticks.

```

However, running this under a fresh Wine 6.11 built (64-bit only;
WINEARCH=win64) on openSUSE (gcc 10.3.0) gives me this:

```
withdll.exe: Starting: `withdll.exe'
withdll.exe:   with `Z:\tmp\wine\simple64.dll'
withdll.exe: DetourCreateProcessWithDllEx failed: 2

```

I have attached the stderr from running
`WINEDEBUG=+timestamp,+process,+tid,+seh,+module wine64 withdll.exe
/d:simple64.dll withdll.exe 2>log.txt` to this case.

I have also uploaded the two binary files (`withdll.exe` and `simple64.dll`) to
Google Drive:

https://drive.google.com/file/d/1DmFHYcm4-quslIAsHV1fwbzi6-JxRZse/view?usp=s...

Note: the source code for these files are MIT licensed, so there should be no
issues in me sharing my binary builds (i.e., I do not consider these files
proprietary).

Is it obvious what's wrong here? Is this a Wine bug or some configuration issue
that I can resolve?

[Bug 51378] Failures with `DetourCreateProcessWithDllEx` for Microsoft's Detours Library

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=51378

--- Comment #1 from Andrew V. Jones  ---
Hi,

Sorry; I should have followed https://wiki.winehq.org/Bugs and tried *latest
development version*. I guess this can be closed.

I just tried this with
https://github.com/wine-mirror/wine/commit/0ec555e58ea9d5b33f4c825e96965ad0c...
and it works!

Does anyone know if this was an intentional or accidental fix?

Cheers,

Andrew

[Bug 51378] Failures with `DetourCreateProcessWithDllEx` for Microsoft's Detours Library

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=51378

Roman Pišl  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #2 from Roman Pišl  ---
Can you find out which commit fixed this?

[Bug 51378] Failures with `DetourCreateProcessWithDllEx` for Microsoft's Detours Library

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=51378

--- Comment #3 from Andrew V. Jones  ---
No joke, this appears to have been fixed in
0ec555e58ea9d5b33f4c825e96965ad0cb15d00f (i.e., HEAD as of starting the `git
bisect`).

I `git bisect`ed in reverse, so 0ec555e58ea was bad (because it worked) and
b3e9d0f88c5 was good (because it didn't), and I asked `git bisect` to tell me
the first commit that was bad.

I let it run and `git` said:

```
0ec555e58ea9d5b33f4c825e96965ad0cb15d00f is the first bad commit

```

so I thought it had a bug in my script, but nope.

In 9a56ba03368, I get:

```
withdll.exe: Starting: `withdll.exe'
withdll.exe:   with `Z:\mnt\baranem\wine\test\simple64.dll'
withdll.exe: DetourCreateProcessWithDllEx failed: 2

```

but with 0ec555e58ea it works!

[Bug 51378] Failures with `DetourCreateProcessWithDllEx` for Microsoft's Detours Library

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=51378

--- Comment #4 from Andrew V. Jones  ---
Is there any way to add a test to avoid this regressing? Happy to help if
there's some easy-to-follow steps for writing such a test.

[Bug 51378] Failures with `DetourCreateProcessWithDllEx` for Microsoft's Detours Library

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=51378

--- Comment #5 from Roman Pišl  ---
So it is a pure coincidence but it is not much surprising as the library is a
kind of interceptor. The problem is described in bug 50993 and bug 44893.
Looking at Detour, it has similar algorithm in detour_alloc_region_from_lo,
detour_alloc_region_from_hi and FindAndAllocateNearBase and for the tests you
could start from here.

[Bug 51378] Failures with `DetourCreateProcessWithDllEx` for Microsoft's Detours Library

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=51378

Gijs Vermeulen  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                URL|                            |https://web.archive.org/web
                   |                            |/20210724090119if_/https://
                   |                            |doc-0s-70-docs.googleuserco
                   |                            |ntent.com/docs/securesc/ha0
                   |                            |ro937gcuc7l7deffksulhg5h7mb
                   |                            |p1/glivhv3bfeamvf2ft63fjqca
                   |                            |h144csr4/1627117275000/1452
                   |                            |2673306128007978/*/1DmFHYcm
                   |                            |4-quslIAsHV1fwbzi6-JxRZse?e
                   |                            |=download
           Keywords|                            |download, source

--- Comment #6 from Gijs Vermeulen  ---
So can we close this FIXED then? With the binaries from the OP, I get the same
results as Windows using current wine-git.

Adding a stable download.

$ sha1sum wine.tar.gz
8d509ff7d038c716b26782f9efb2edb0cbc92893  wine.tar.gz

$ du -sh wine.tar.gz
160K    wine.tar.gz

[Bug 51378] Failures with `DetourCreateProcessWithDllEx` for Microsoft's Detours Library

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=51378

Gijs Vermeulen  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #7 from Gijs Vermeulen  ---
(In reply to Gijs Vermeulen from comment #6)

No reply to this in almost 4 years and OP mentioned this being fixed in Comment
#3, so marking FIXED.

[Bug 51378] Failures with `DetourCreateProcessWithDllEx` for Microsoft's Detours Library

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=51378

Alexandre Julliard  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #8 from Alexandre Julliard  ---
Closing bugs fixed in 10.6.