[Bug 54015] New: Accessing shared memory from Linux
https://bugs.winehq.org/show_bug.cgi?id=54015 Bug ID: 54015 Summary: Accessing shared memory from Linux Product: Wine Version: unspecified Hardware: x86-64 OS: Linux Status: UNCONFIRMED Severity: normal Priority: P2 Component: kernel32 Assignee: wine-bugs(a)winehq.org Reporter: berarma(a)hotmail.com Distribution: --- Some Windows games and applications share some internals through shared memory with external tools. Shared memory is preferred over sockets because it adds no overhead with no latency. It's mostly used in driving and flying simulators to provide visual overlays, data on additional displays, telemetry, rig movement, etc. from external tools. There's also scientific apps using it for similar purposes or general inter-process communication. It's common to use tagged shared memory blocks that external tools can then access using the already known tag. Here's an example from a Python application for rFactor 2: self._rf2_tele = mmap.mmap(0, ctypes.sizeof(rF2Telemetry), f"$rFactor2SMMP_Telemetry${input_pid}") self._rf2_scor = mmap.mmap(0, ctypes.sizeof(rF2Scoring), f"$rFactor2SMMP_Scoring${input_pid}") self._rf2_ext = mmap.mmap(0, ctypes.sizeof(rF2Extended), f"$rFactor2SMMP_Extended${input_pid}") self._rf2_ffb = mmap.mmap(0, ctypes.sizeof(rF2ForceFeedback), "$rFactor2SMMP_ForceFeedback$") Allowing these shared memory blocks to be accessed from Linux would allow us to port some of these tools or even implement or own. Then these tools could access resources or hardware that isn't available from inside the Wine prefix. Some of these tools also fail to run on Wine. Since I guess Wine is already using Linux shared memory mechanism to implement Windows shared memory, the blocks are already available at /dev/shm but they use generic names that can't be associated to the tag used on Windows. Having these shared memory blocks created in the prefix directory with a name containing the tag used on creation would allow us to use it from our native applications. Or any other way to find them using the tags used on Windows. Thanks for your attention. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54015 Fabian Maurer <dark.shadow4(a)web.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dark.shadow4(a)web.de --- Comment #1 from Fabian Maurer <dark.shadow4(a)web.de> --- If I'm not completely wrong, making a winelib application allows you to use windows functions and unix functions at the same time. Would that be a solution? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54015 --- Comment #2 from Bernat Arlandis <berarma(a)hotmail.com> --- (In reply to Fabian Maurer from comment #1)
If I'm not completely wrong, making a winelib application allows you to use windows functions and unix functions at the same time. Would that be a solution?
Say I want to port a Python application that is already fully cross-platform except for the incompatible mmap calls. This is a real case we are trying to solve. Writting a new application when there's already one that works isn't a good solution. Is there a way we could use winelib from Python? I need to see an example of the Python lines above ported to Winelib using Python if possible. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54015 --- Comment #3 from Austin English <austinenglish(a)gmail.com> --- (In reply to Bernat Arlandis from comment #2)
(In reply to Fabian Maurer from comment #1)
If I'm not completely wrong, making a winelib application allows you to use windows functions and unix functions at the same time. Would that be a solution?
Say I want to port a Python application that is already fully cross-platform except for the incompatible mmap calls. This is a real case we are trying to solve.
It would be helpful if you could name the application, particularly if source is available. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54015 Eric Pouech <eric.pouech(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |eric.pouech(a)gmail.com --- Comment #4 from Eric Pouech <eric.pouech(a)gmail.com> --- (In reply to Bernat Arlandis from comment #2)
(In reply to Fabian Maurer from comment #1)
If I'm not completely wrong, making a winelib application allows you to use windows functions and unix functions at the same time. Would that be a solution?
Say I want to port a Python application that is already fully cross-platform except for the incompatible mmap calls. This is a real case we are trying to solve.
Writting a new application when there's already one that works isn't a good solution.
Is there a way we could use winelib from Python? I need to see an example of the Python lines above ported to Winelib using Python if possible.
that's not an easy task: - python likely requires you to load a shared lib from which you can export functions callable in python - then you need to insert wine code in that shared lib. Depending on exec models and lots of other quirks, it may or may not work. - if it doesn't, the trick would be to split the effort in two. shared lib on one hand (that does the mapping - note that wine doesn't use shm but (temporary) file mappings instead) and wine app on the other one (that's requested by shared lib to get the relevant (temp) file by its name) Perhaps another wAYould be to simply use the windows' python exec (didn't try it,and it may w -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54015 --- Comment #5 from Eric Pouech <eric.pouech(a)gmail.com> --- to finish previous command: - another alternative would be to run the windows python interpreter under wine (never tested it, so could be another can of worms); but should handle these mappings -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54015 --- Comment #6 from Bernat Arlandis <berarma(a)hotmail.com> --- (In reply to Austin English from comment #3)
(In reply to Bernat Arlandis from comment #2)
(In reply to Fabian Maurer from comment #1)
If I'm not completely wrong, making a winelib application allows you to use windows functions and unix functions at the same time. Would that be a solution?
Say I want to port a Python application that is already fully cross-platform except for the incompatible mmap calls. This is a real case we are trying to solve.
It would be helpful if you could name the application, particularly if source is available.
Sure, sorry about that. It's here: https://github.com/s-victor/TinyPedal (In reply to Eric Pouech from comment #4)
that's not an easy task: - python likely requires you to load a shared lib from which you can export > functions callable in python - then you need to insert wine code in that shared lib. Depending on exec models and lots of other quirks, it may or may not work. - if it doesn't, the trick would be to split the effort in two. shared lib on one hand (that does the mapping - note that wine doesn't use shm but (temporary) file mappings instead) and wine app on the other one (that's requested by shared lib to get the relevant (temp) file by its name)
Oh, I thought it used shm. Is there some example of something like this in use? (In reply to Eric Pouech from comment #5)
to finish previous command: - another alternative would be to run the windows python interpreter under wine (never tested it, so could be another can of worms); but should handle these mappings
I tried a compiled exe that is distributed in the project but it fails. Since we're so close the having a working native port I thought it would be easier to try to solve the shared memory issue and at the same time i would be useful for other projects too. Thanks for the responses. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54015 Zeb Figura <z.figura12(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |z.figura12(a)gmail.com --- Comment #7 from Zeb Figura <z.figura12(a)gmail.com> --- (In reply to Bernat Arlandis from comment #0)
Since I guess Wine is already using Linux shared memory mechanism to implement Windows shared memory, the blocks are already available at /dev/shm but they use generic names that can't be associated to the tag used on Windows.
Not exactly; we use anonymous shared mappings instead.
Having these shared memory blocks created in the prefix directory with a name containing the tag used on creation would allow us to use it from our native applications. Or any other way to find them using the tags used on Windows.
This is not impossible in some cases, but it's also harder than it sounds. There's a bit of an impedance mismatch between NT and POSIX wrt naming: notably POSIX has a NAME_MAX restriction which doesn't apply to Windows; Windows allows all characters (including \0 I think?) except backslashes and Unix all characters except \0 and forward slashes; Windows uses UTF-16 while Unix uses a non-constant character set. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54015 --- Comment #8 from Bernat Arlandis <berarma(a)hotmail.com> --- (In reply to Zeb Figura from comment #7)
(In reply to Bernat Arlandis from comment #0)
Since I guess Wine is already using Linux shared memory mechanism to implement Windows shared memory, the blocks are already available at /dev/shm but they use generic names that can't be associated to the tag used on Windows.
Not exactly; we use anonymous shared mappings instead.
Having these shared memory blocks created in the prefix directory with a name containing the tag used on creation would allow us to use it from our native applications. Or any other way to find them using the tags used on Windows.
This is not impossible in some cases, but it's also harder than it sounds. There's a bit of an impedance mismatch between NT and POSIX wrt naming: notably POSIX has a NAME_MAX restriction which doesn't apply to Windows; Windows allows all characters (including \0 I think?) except backslashes and Unix all characters except \0 and forward slashes; Windows uses UTF-16 while Unix uses a non-constant character set.
I understand. Thanks for the insight. They don't need to be the exact same names on Linux. We could use a translation scheme like escaping non-supported characters which would gives us unique and universal translations so we always know how the name would be translated on Linux for every machine. It's not something the users have to read, it's just an id in the code to access a resource so legibility isn't an issue. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54015 --- Comment #9 from Fabian Maurer <dark.shadow4(a)web.de> --- Then again, can't you use a winelib part as middleman? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54015 --- Comment #10 from Bernat Arlandis <berarma(a)hotmail.com> --- (In reply to Fabian Maurer from comment #9)
Then again, can't you use a winelib part as middleman?
I'll learn to use winelib and I'll see how it can work. Thanks. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54015 Alex Barrero <alex.brrsclnt(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |alex.brrsclnt(a)gmail.com --- Comment #11 from Alex Barrero <alex.brrsclnt(a)gmail.com> --- Will winelib work with a python app? I though that is intended for C/C++ apps -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54015 --- Comment #12 from Bernat Arlandis <berarma(a)hotmail.com> --- (In reply to Alex Barrero from comment #11)
Will winelib work with a python app? I though that is intended for C/C++ apps
The middleman would be a command/app probably written in C/C++ that sits between the Python app and the Proton game. The middleman can copy from Wine shared memory to Linux shared memory so native applications can access the data. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54015 redartofcode(a)gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |redartofcode(a)gmail.com -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
participants (1)
-
WineHQ Bugzilla