I've been trying to come up with a way to solve the problem of creating absolute symlinks inside a WINEPREFIX, since I believe this is now the last barrier for the Junction Point/Symlink patches*. The current staging patch includes the prefix path in the absolute symlink, which creates obvious problems if the prefix is ever moved to a new location.
A proposal a while back was to use "/proc/self/cwd" as part of the symlink and force wine to always use the prefix folder as the "real" working directory. This has the downside that the symlinks will break outside of Wine, but got me thinking about other possible ways to make the absolute symlinks work outside of Wine. So, I put together a small tool that builds a chroot for the prefix and creates a special root folder "??" with bind mounts that allows symlinks to follow a semi-NT style (e.g. '/??/C:/windows/system32' instead of '??\C:\windows\system32'). This technique wound up working remarkably well, with the obvious downside that it requires additional administrative privileges for the "launcher" app.
This idea has a logical extension to drop the need for admin rights: use LD_PRELOAD and wrap essential path resolution functions. While conceptually straightforward, this is very challenging to do in practice. Fortunately, we are not actually trying to create a chroot - we just need to inject a fake "??" folder at the root mount point. After quite a bit of trial and error I've managed to put together a "wineprefix" tool that does this correctly for both Wine and bash, a couple interesting examples are attached (examples.txt).
I'm curious what people think about this approach. You clearly need to run "wineprefix" for the symlinks to be valid, but once you've done that everything "just works". So, what do people think? Is this worth turning into a tool for Wine or is it completely insane?
Best, Erich
* sans some rebasing work that recently got introduced
Doesn't it defeat the purpose of using symlinks that resolve properly if they only work inside wine processes?
On Sat, May 16, 2020 at 5:03 PM Esme Povirk (they/them) < vincent@codeweavers.com> wrote:
Doesn't it defeat the purpose of using symlinks that resolve properly if they only work inside wine processes?
The way this is put together you can run "wineprefix <program> <arguments>" to allow these symlinks to resolve properly. The default behavior (no arguments) is to open bash, and this behavior propagates such that applications run within the shell work the same way, a couple examples: === ehoover@lappy:~$ wineprefix ehoover@lappy:/??/C:$ echo "TEST" | gpg -c --batch --yes --passphrase "test" --output /??/C:/TEST/test.gpg - ehoover@lappy:/??/C:$ exit ehoover@lappy:~$ wineprefix gpg -d /??/C:/TEST/test.gpg gpg: AES256 encrypted data gpg: encrypted with 1 passphrase TEST === So, the symlinks do work for non-wine processors (when launched with the special LD_PRELOAD).
Also, this is only necessary for within-prefix links - any symlinks targeting outside the prefix would not require any special treatment.
Best, Erich
I think this would be too much complexity to be worth it. By the time I learn as a user that my Wine-created symlinks will only work using some special command, and manually launch whatever program I want to use them in, I've already been inconvenienced more than I would be by a broken symlink. I'd also worry about the potential for compatibility problems with applications launched using wine's LD_PRELOAD (by winebrowser for example).
I'm also not clear on why the symlinks need to resolve correctly *inside wine processes*. As long as we only open them using the win32 api, the win32 api can interpret those files however we need it to.
Wine should control any file access that needs to work with symlinks it creates, especially with the push towards converting Wine's dlls to be built as PE. Therefore, it should be able to do the right thing in its own code and not have to hook itself or any native libraries it loads. "The right thing" might mean interpreting a special /??/ path differently, replacing a wineprefix embedded in the target with the current wineprefix, or reading the target using something other than readlink(), but whatever it is should be equally doable inside Wine without hooking.
On Sat, May 16, 2020 at 8:34 PM Esme Povirk (they/them) vincent@codeweavers.com wrote:
I think this would be too much complexity to be worth it. By the time I learn as a user that my Wine-created symlinks will only work using some special command, and manually launch whatever program I want to use them in, I've already been inconvenienced more than I would be by a broken symlink. I'd also worry about the potential for compatibility problems with applications launched using wine's LD_PRELOAD (by winebrowser for example).
Well, a choice could be to embed the absolute path such that as long as the prefix isn't moved that it still works. However, this gets back into the category of "why don't we just rewrite the symlinks if the prefix gets moved" that we talked about at WineConf that people didn't seem really keen on.
I'm also not clear on why the symlinks need to resolve correctly *inside wine processes*. As long as we only open them using the win32 api, the win32 api can interpret those files however we need it to.
It's not strictly necessary for Wine to be run like this, though you would still need the wineserver to deal with the case where a path contains an NT symlink along the way.
Wine should control any file access that needs to work with symlinks it creates, especially with the push towards converting Wine's dlls to be built as PE. Therefore, it should be able to do the right thing in its own code and not have to hook itself or any native libraries it loads. "The right thing" might mean interpreting a special /??/ path differently, replacing a wineprefix embedded in the target with the current wineprefix, or reading the target using something other than readlink(), but whatever it is should be equally doable inside Wine without hooking.
This was meant as a demo, but if we were to do something like this then you would implement most of the real work in the wineserver so that the LD_PRELOAD is only needed for other programs.
On Sun, May 17, 2020 at 6:48 AM Gabriel Ivăncescu gabrielopcode@gmail.com wrote:
... Can't you just make all the absolute symlinks within a prefix actually relative, by converting them? Then use a special pattern to recognize them as "absolute" so Wine knows to report them as such (any combination of / and . ).
We already have the "type" encoded in the staging patches, the link target always begins with either "." (relative) or "/" (absolute) followed by the reparse tag encoded as dots and slashes, for example: IO_REPARSE_TAG_SYMLINK: //././/////////////////////////.//./ For symlinks this is then followed by an "is directory" flag because of annoying Windows reasons.
The only downside is that they will break if the symlinks themselves are moved within the prefix by programs outside of Wine. But then again, so will they with your current approach, if they don't run the 'wineprefix' tool. ...
And if they or their parent is moved by programs within Wine, unless you modify MoveFile in a very nontrivial way. If you move the parent folder of the symlink then the symlink breaks, which means that whenever you move a folder you would need to check _all_ the contents for symlinks and then go through and rewrite them all.
I still think the easiest thing to do would be to treat these links as absolute links and make it easy to rewrite the portion corresponding to the prefix if the prefix is moved, but people didn't really seem to like that. It just seems to me like this way is easier and more robust to external applications (doesn't require special tools unless you move a prefix, and we could conceivably provide a tool for the expressed purpose of moving a prefix). If you look at how the staging patches are doing things now: <Relative|Absolute><Reparse Tag><Directory|File><Path to Target> it seems like this can be extended pretty easily such that "Path to Target" gets split into: <Prefix><Magic Pattern><Prefix-specific Path> Then when the wineserver opens something and it fails all we need to do is a quick check for symlinks and rewrite the prefix portion of the path.
Best, Erich
Fundamentally I guess we're trying to simultaneously satisfy the constraints of
(1) make NT symlinks, regardless of type, behave like Unix symlinks, to any host filesystem utilities
(2) make absolute symlinks still work when the prefix is moved, without any additional work
Which, barring some brilliant idea, I think just isn't possible, not without rewriting how said host utilities work, either by preloading our own bits of glibc, or maybe introducing some mount-point-like API to the kernel.
Personally, I think dropping constraint (2) is preferable to any hack involving preloading. Manually fixing up symlinks after a prefix move feels equivalent to manually prepending a wrapper before any filesystem operation, but it would have to be done far less often.
I also think it makes more sense to drop constraint (2) than constraint (1). I suspect moving a prefix is very rare in practice (though to be sure I have done it myself a couple times), whereas accessing the Win32 filesystem from outside is probably quite common. Additionally, it ends up being far less work to rely on host symlink resolution than to have to implement it ourselves.
On Tue, May 19, 2020 at 9:45 AM Zebediah Figura z.figura12@gmail.com wrote:
Fundamentally I guess we're trying to simultaneously satisfy the constraints of
(1) make NT symlinks, regardless of type, behave like Unix symlinks, to any host filesystem utilities
(2) make absolute symlinks still work when the prefix is moved, without any additional work
Which, barring some brilliant idea, I think just isn't possible, not without rewriting how said host utilities work, either by preloading our own bits of glibc, or maybe introducing some mount-point-like API to the kernel.
Thanks Zeb, I think this is a nice, succinct summary of the situation.
Personally, I think dropping constraint (2) is preferable to any hack involving preloading. Manually fixing up symlinks after a prefix move feels equivalent to manually prepending a wrapper before any filesystem operation, but it would have to be done far less often.
This is my feeling as well, but when everyone was chatting at WineConf people didn't seem to like this idea. Granted, this conversation happened at the bar and I may be me misremembering the situation.
I also think it makes more sense to drop constraint (2) than constraint (1). I suspect moving a prefix is very rare in practice (though to be sure I have done it myself a couple times), whereas accessing the Win32 filesystem from outside is probably quite common. Additionally, it ends up being far less work to rely on host symlink resolution than to have to implement it ourselves.
I have done a lot of relocating prefixes for running automated builds, and I personally wouldn't mind using a tool to backup/move prefixes, but I cannot really speak for others. As I mentioned, it doesn't seem to me like it would be difficult to make it easy for such a tool to operate - we would just need to add a minor tweak to the way the symlink target is stored so that portion corresponding to the prefix is obvious. If we were feeling particularly kind then we could distribute such a tool as a simple shell script without too much trouble.
Best, Erich
On 17/05/2020 01:48, Erich E. Hoover wrote:
I've been trying to come up with a way to solve the problem of creating absolute symlinks inside a WINEPREFIX, since I believe this is now the last barrier for the Junction Point/Symlink patches*. The current staging patch includes the prefix path in the absolute symlink, which creates obvious problems if the prefix is ever moved to a new location.
Hi Erich,
Can't you just make all the absolute symlinks within a prefix actually relative, by converting them? Then use a special pattern to recognize them as "absolute" so Wine knows to report them as such (any combination of / and . ).
The only downside is that they will break if the symlinks themselves are moved within the prefix by programs outside of Wine. But then again, so will they with your current approach, if they don't run the 'wineprefix' tool.
And in worst case you can still make such a tool to handle those symlinks if you need to move them outside of Wine, for some reason...
Normal Unix symlinks created outside of Wine shouldn't be affected.
-
Erich E. Hoover -
Esme Povirk (they/them) -
Gabriel Ivăncescu -
Zebediah Figura