POSIX shared memory between regular Linux and Wine processes?

[Sebastian M. Ernst]

Hi all,

I could not find documentation and/or a working example, but a couple of
"unfinished" discussions on the subject: Is it feasible / possible in
any way, shape or form to literally share memory between a regular Linux
process and a Wine process (on top of Linux) via POSIX shared memory? I
am specifically talking about the `shmat`, `shmctl`, `shmdt` and
`shmget` system calls. Has anyone around here ever done something like this?

Thanks,
Sebastian


[1] https://en.wikipedia.org/wiki/Shared_memory#Support_on_Unix-like_systems

[Paul Gofman]

Hi Sebastian,

    can you share any details on the supposed use case for that?

    Windows creates shared memory through file mappings [1]. File
mappings are translated into the analogous Unix file mappings (mmap()
allows to allocate shared memory backed by a file). For "anonymous"
named shared memory objects Wine creates a temporary file, otherwise it
is the file specified by the application. So if you want to share the
memory which is shared by the Windows app through named file you can
just share that memory with mmap() with that file.
    If you are thinking of somehow working with the app's private memory
as shared, it is not immediatley clear to me how that makes sense.

Regards,

    Paul.


1.
https://docs.microsoft.com/en-us/windows/win32/memory/creating-named-shared-...

On 7/14/21 14:06, Sebastian M. Ernst wrote:
> Hi all,
>
> I could not find documentation and/or a working example, but a couple of
> "unfinished" discussions on the subject: Is it feasible / possible in
> any way, shape or form to literally share memory between a regular Linux
> process and a Wine process (on top of Linux) via POSIX shared memory? I
> am specifically talking about the `shmat`, `shmctl`, `shmdt` and
> `shmget` system calls. Has anyone around here ever done something like this?
>
> Thanks,
> Sebastian
>
> [1] https://en.wikipedia.org/wiki/Shared_memory#Support_on_Unix-like_systems

[Sebastian M. Ernst]

Hi Paul,
>     can you share any details on the supposed use case for that?

sure. I use Wine mainly for scientific computing where e.g. part of a
simulation is implemented for Windows only while the "rest" runs
natively on Linux. In Linux processes, I must "simply" call methods in
Windows DLLs and pass pointers (to large multidimensional arrays) to
them with as little as possible overhead. I have developed my own
generic tooling for this for CPython over the years which essentially
serializes the data (or just copies the memory contents) and ships it
back and forth via sockets, trying to keep the madness on both sides in
sync as good as possible:

https://github.com/pleiszenburg/zugbruecke/tree/develop

It works fine but does not really scale well to larger quantities of
data as you can probably imagine.

If I had a relatively easy to handle option of somehow actually sharing
any form of pointer between my Linux processes and my Windows DLL
functions (called via a separate Wine processes), it would open up a
bunch of very interesting possibilities. I am aware of the option of
turning my simulations into "actual Wine apps" which could then access
both Linux and Windows libraries, i.e. no need to share memory between
processes, but this is not exactly a very trivial, sustainable or easy
to distribute solution.
> [...] mmap() [...]

Thanks for the explanation. I have considered playing with an mmap-based
approach directly, maybe even putting the mapped file itself into a
tmpfs or /dev/shm, but I have not had the time for this yet. Long story
short and I may be fundamentally misunderstanding something here: Would
not any approach like this still involve multiple copies of my data or
is there a way to actually have a transparent, common memory space of
some kind? Does not something like this always go at least through the
OS' VFS layer hence slowing down memory access?

Best regards,
Sebastian

[Paul Gofman]

On 7/14/21 15:08, Sebastian M. Ernst wrote:
> Thanks for the explanation. I have considered playing with an mmap-based
> approach directly, maybe even putting the mapped file itself into a
> tmpfs or /dev/shm, but I have not had the time for this yet. Long story
> short and I may be fundamentally misunderstanding something here: Would
> not any approach like this still involve multiple copies of my data or
> is there a way to actually have a transparent, common memory space of
> some kind? Does not something like this always go at least through the
> OS' VFS layer hence slowing down memory access?

If your Windows DLL cooperates you just need to create file mappings on
Windows DLL side with a named file which can also be opened on the Unix
side and used with mmap. The memory shared this way is just a shared
memory (just backed by the file which file might be updated with some
delay but not the memory mapped to the processes). It has no principle
differences to POSIX shm you linked in this regard. Please note that
while you can use shared memory this way you might still not be able to
pass pointers directly (regardless of the Unix or Windows API you use to
share the memory). The memory, while physically shared, might be mapped
to different addresses in different processes, so the pointers still
won't be valid. You can try to overcome that by mapping the shared
memory at fixed addresses on both Windows (MapViewOfFileEx accepts
mapping address) and mmap() which also accepts the address (Linux also
has MAP_FIXED_NOREPLACE flag nowadays which might be helpful in this
case). But this can fail on either side as the selected memory range
might get randomly busy by something else in the process.