Does wine perform exact same memory allocations than Windows?

[Emanuele Oriani]

Hi wine-devels,

I am trying to port a Windows program, SmartHunter, to Linux (port is 
https://github.com/Emanem/linux-hunter).

This Windows program is a companion app 
(https://github.com/sir-wilhelm/SmartHunter) for a game (Monster Hunter 
: World). This companion app relies on scanning memory and finding 
precise memory patterns and then, based on such patterns, dereference 
pointers (even multiple levels) and access structures in memory.

Made up example:

1. find memory pointer which matches 48 8B 0D ?? ?? ?? ?? E8 (where ?? 
can be any byte)
2. interpret the four ?? ?? ?? ?? as a 32 bit pointer
3. dereference what pointed as a utf8 string (or a double or int or ...) 
or recursively dereference up until you reach desired structure in memory

In terms of porting this program, I have my port running natively on 
Linux against a wine process:

- I execute "Monster Hunter : World" via wine/proton
- I run my ported app and then it scans the memory of the wine process

When porting this program I have been able to resolve and navigate some 
structures, but some others I can't. I can find some complex patterns 
(such as "48 8B 0D ?? ?? ?? ?? E8 ?? ?? ?? ?? 48 8B 0D ?? ?? ?? ?? E8 ?? 
?? ?? ?? 48 8B 05 ?? ?? ?? ?? 83 A0 ?? ?? ?? ?? ?? C6 43" or "48 8B 0D 
?? ?? ?? ?? B2 01 E8 ?? ?? ?? ?? C6 83 ?? ?? ?? ?? ?? 48 8B 0D"), but 
when I start "navigating" those and dereference some alleged pointers I 
end up outside of the process memory spaces.
Furthermore I seem to have found that for some types (namely utf8 
strings) seems like wine allocates 1 more byte than Windows, hence 
somehow deviating from expected memory layout.

Am I correct in thinking that wine allocations may be slightly different 
than Windows (i.e. larger)?

This being the case it would change the expected layout.

Thanks in advance,
Emanuele

[Stefan Dösinger]

> Am 27.05.2020 um 09:00 schrieb Emanuele Oriani <emaentra(a)ngi.it>:
>
> When porting this program I have been able to resolve and navigate some structures, but some others I can't. I can find some complex patterns (such as "48 8B 0D ?? ?? ?? ?? E8 ?? ?? ?? ?? 48 8B 0D ?? ?? ?? ?? E8 ?? ?? ?? ?? 48 8B 05 ?? ?? ?? ?? 83 A0 ?? ?? ?? ?? ?? C6 43" or "48 8B 0D ?? ?? ?? ?? B2 01 E8 ?? ?? ?? ?? C6 83 ?? ?? ?? ?? ?? 48 8B 0D"), but when I start "navigating" those and dereference some alleged pointers I end up outside of the process memory spaces.
> Furthermore I seem to have found that for some types (namely utf8 strings) seems like wine allocates 1 more byte than Windows, hence somehow deviating from expected memory layout.
> What you are doing here is very fragile, even going from Windows to Windows. I realize it is a common thing to do for game mods though if the game does not provide an API for such modifications.

How reliably memory patterns are replicated between Wine and Windows and even two different Windows versions depends on how the allocations are made. If you are looking up pointers into the game's code in its DLLs and EXE files they are very similar because the PE file is mmap'ed into the processes' address space. You have good chances of the absolute addresses to be identical.

If the game allocates a big blob of Heap memory in one go and fills it with data you should also be lucky. If there are multiple independent heap allocations done by the game the patterns will start to look differently. Wine will not allocate smaller memory blobs than requested, but a heap allocation may be slightly larger, placed in different areas of the address space, etc. The exact details not only depend on Wine, but also on the Linux kernel, linux libs etc.

Things will get even more spotty if the actual memory allocations are done by some Windows API functions. I don't know the string APIs in detail, so the following example is just a hypothetical one: If the game loads data from an XML file we pass the heavy lifting to the Linux libxml2 library. Its internal workings are different from microsoft's msxml.dll so the layout of the loaded file will not look alike at all.

You can try to look into some observable allocation properties with functions like HeapSize and VirtualQuery. One thing worth exploring is finding memory allocations not by searching for magic patterns in memory but hooking functions that the game uses to load the data. It may or may not work better.

[Emanuele Oriani]

Hi Stefan,

Thanks for replying back and confirming about different memory 
allocations/patters.
This corroborates my thinking and for sure makes the porting of such 
companion apps non trivial.

| What you are doing here is very fragile, even going from Windows to 
Windows. I realize it is a common thing to do for game mods though if 
the game does not provide an API for such modifications.

I'm fully aware of the fragility and the modding community is also 
conscious that updates in the the game may 'break' the logic, even on 
Windows; in this particular case it's just about reading memory 
thankfully (primarily traversing pointers).

| If you are looking up pointers into the game's code in its DLLs and 
EXE files they are very similar because the PE file is mmap'ed into the 
processes' address space. You have good chances of the absolute 
addresses to be identical.

Some addresses are identical, some other are not, but if it's about 
following pointers, I think we're good - after all a pointer is a 
pointer both in wine and native Windows.
Basically if one can find a valid AoB/base address, and then solely 
operate with pointers, then the Windows logic may be ported with just 
some trouble but not impossible.

Everything else, agree 100%.

Emanuele