[Bug 50281] New: fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

Bug ID: 50281
           Summary: fsync patch for wine 6.0-rc1 (without staging)
           Product: Wine
           Version: 6.0-rc1
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: enhancement
          Priority: P2
         Component: wineserver
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

Created attachment 68830
  --> https://bugs.winehq.org/attachment.cgi?id=68830
fsync patch that applies to vanilla wine 6.0-rc1

I have created a new fsync patch that can apply directly to wine 6.0-rc1
without needing esync from staging.

I have tested this patch applied to the latest wine sources (6.0-rc1) as of 7th
December 2020 using Star Citizen 3.11.

I have included it here for you to review for your convenience to review
whether to include it in a future wine version.

Current testing from myself and others in the Star Citizen LUG org when using
fsync enabled wine and kernels are resulting in a much smoother experience.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #1 from Mike Scott  ---
FYI, I have been updating the LUG org with fsync patches on this thread:

https://robertsspaceindustries.com/spectrum/community/LUG/forum/149/thread/f...

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

Zebediah Figura  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |WONTFIX
                 CC|                            |[e-mail]

--- Comment #2 from Zebediah Figura  ---
"fsync" will not be included in upstream Wine.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #3 from Austin English  ---
Closing.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #4 from Austin English  ---
Closing.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

Shmerl  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #5 from Shmerl  ---
(In reply to Zebediah Figura from comment #2)
> "fsync" will not be included in upstream Wine.

May be this it outdated, but is this still the case?

futex2 just landed in the upcoming Linux 5.16 and I thought Wine will
eventually make use of it, but I saw this bug and now I'm confused whether
using futex2 is still the plan or not. And what about Wine-staging?

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #6 from Zebediah Figura  ---
(In reply to Shmerl from comment #5)
> (In reply to Zebediah Figura from comment #2)
> > "fsync" will not be included in upstream Wine.
>
> May be this it outdated, but is this still the case?
>
> futex2 just landed in the upcoming Linux 5.16 and I thought Wine will
> eventually make use of it, but I saw this bug and now I'm confused whether
> using futex2 is still the plan or not. And what about Wine-staging?

Yes, it is still the case. The lack of a kernel interface is (was) not the only
reason why fsync will not be included.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #7 from Shmerl  ---
(In reply to Zebediah Figura from comment #6)
> Yes, it is still the case. The lack of a kernel interface is (was) not the
> only reason why fsync will not be included.

And what about Wine staging? So all that effort by Collabora in the end didn't
end up being successful? I always thought it was intended for Wine.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #8 from Zebediah Figura  ---
(In reply to Shmerl from comment #7)
> (In reply to Zebediah Figura from comment #6)
> > Yes, it is still the case. The lack of a kernel interface is (was) not the
> > only reason why fsync will not be included.
>
> And what about Wine staging? So all that effort by Collabora in the end
> didn't end up being successful? I always thought it was intended for Wine.

I am doubtful that it will be accepted into wine-staging either.

I believe that Collabora's effort was for the benefit of Steam Proton and other
forks or community projects, as well as for some other consumers who wanted
vectored futex waits.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #9 from Shmerl  ---
(In reply to Zebediah Figura from comment #8)
> I am doubtful that it will be accepted into wine-staging either.

Can you please explain why it can't be accepted in either staging or regular
Wine or point out to some thread that already explains that to avoid repeating
it? It's kind of disheartening for users of Wine who don't use Proton since it
will require applying these patches manually if won't be part of staging
either.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #10 from Zebediah Figura  ---
(In reply to Shmerl from comment #9)
> (In reply to Zebediah Figura from comment #8)
> > I am doubtful that it will be accepted into wine-staging either.
>
> Can you please explain why it can't be accepted in either staging or regular
> Wine or point out to some thread that already explains that to avoid
> repeating it? It's kind of disheartening for users of Wine who don't use
> Proton since it will require applying these patches manually if won't be
> part of staging either.

Neither esync nor fsync will be accepted into upstream wine for the following
reasons:

* they rely on use of shared memory to retain object state, which is not robust
against misbehaving processes (and hence is not secure either);

* they do not offer full compatibility with the windows API and in practice
break many applications.

And no, neither one of these can be fixed within the current design, or I would
have done so already.

fsync is unlikely to be accepted into wine-staging because it represents a
significant additional maintenance burden for practically no benefit. In
particular the number of applications that it is known to help can be counted
on one hand. Furthermore esync is already a very significant maintenance
burden; it has been easily the worst patch set in wine-staging since its
induction.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #11 from Shmerl  ---
(In reply to Zebediah Figura from comment #10)
> fsync is unlikely to be accepted into wine-staging because it represents a
> significant additional maintenance burden for practically no benefit. In
> particular the number of applications that it is known to help can be
> counted on one hand. Furthermore esync is already a very significant
> maintenance burden; it has been easily the worst patch set in wine-staging
> since its induction.

Thanks for the explanation. At least that's good to know that there aren't many
applications that actually benefit from it.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #12 from Shmerl  ---
Though since Wine staging already has esync patches maintenance burden, would
it make sense to remove these patches from staging and at least replace them
with fsync ones, or there there is no point in doing it?

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #13 from Zebediah Figura  ---
(In reply to Shmerl from comment #12)
> Though since Wine staging already has esync patches maintenance burden,
> would it make sense to remove these patches from staging and at least
> replace them with fsync ones, or there there is no point in doing it?

No. fsync is less widely available than esync (e.g. it requires a much newer
kernel, or I suppose I should say will require since IIRC sys_futex_waitv isn't
even merged yet.) Additionally I believe fsync has been known to provide less
stable performance in some cases, so it's not even clearly "at least as good".

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #14 from Shmerl  ---
(In reply to Zebediah Figura from comment #13)
> fsync is less widely available than esync (e.g. it requires a much newer
> kernel, or I suppose I should say will require since IIRC sys_futex_waitv
> isn't even merged yet.) Additionally I believe fsync has been known to
> provide less stable performance in some cases, so it's not even clearly "at
> least as good".

Kernels usage will catch up (futex2 is in 5.16 which is coming out shortly),
but actual usefulness is more critical. If esync is in general not worse and
even better than using fsync / futex2 why did they even bother adding that to
the kernel?

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #15 from Zebediah Figura  ---
(In reply to Shmerl from comment #14)
> If esync is in general not worse and
> even better than using fsync / futex2 why did they even bother adding that
> to the kernel?

In a few cases fsync does offer a non-negligible performance improvement. I'm
not sure it's ever been exactly established that fsync has less stable
performance consistently across test environments, either; I've just seen
multiple reports that it's worse for some applications.

And, as mentioned, I am dimly aware of other uses for vectored futex wait, or
at least intended uses, outside of Wine.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

Sveinar Søpler  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #16 from Sveinar Søpler  ---
(In reply to Zebediah Figura from comment #15)
> In a few cases fsync does offer a non-negligible performance improvement.
> I'm not sure it's ever been exactly established that fsync has less stable
> performance consistently across test environments, either; I've just seen
> multiple reports that it's worse for some applications.
>
> And, as mentioned, I am dimly aware of other uses for vectored futex wait,
> or at least intended uses, outside of Wine.

Fsync patches could possibly be improved more by those that know wine code the
best - WineHQ devs :)

But yes, it is not without flaws, and it is hard to quantify real performance
benefits. When using Wine for games - every FPS counts, and every uS less
latency counts, but the cost probably being as you say a maintenance burden.

If native Linux games starts using this call and can show improvements by doing
so, maybe the argument can be had again in the future :)

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

[e-mail] changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #17 from [e-mail] ---
(In reply to Zeb Figura from comment #10)
> I am doubtful that it will be accepted into wine-staging either.
> Furthermore esync is already a very significant
> maintenance burden; it has been easily the worst patch set in wine-staging
> since its induction.

What are the future plans for esync then?
Will it stay in staging or might be remove at some point?

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #18 from Zeb Figura  ---
(In reply to roidal from comment #17)
> (In reply to Zeb Figura from comment #10)
> > I am doubtful that it will be accepted into wine-staging either.
> > Furthermore esync is already a very significant
> > maintenance burden; it has been easily the worst patch set in wine-staging
> > since its induction.
>
> What are the future plans for esync then?
> Will it stay in staging or might be remove at some point?

The current hope is to obviate esync with a new Linux kernel driver capable of
performantly emulating NT synchronization primitives. If successful, it should
provide equal or better performance, while matching the compatibility and
robustness requirements of upstream. In this case we will no longer be keeping
esync in wine-staging.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #19 from [e-mail] ---
(In reply to Zeb Figura from comment #18)
> The current hope is to obviate esync with a new Linux kernel driver capable
> of performantly emulating NT synchronization primitives.

What is the actual approach to that?
I was able to find an old mail to ntsync/winesync but sadly nothing about it's
progress.
Is it still in developement?

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #20 from Zeb Figura  ---
(In reply to roidal from comment #19)
> (In reply to Zeb Figura from comment #18)
> > The current hope is to obviate esync with a new Linux kernel driver capable
> > of performantly emulating NT synchronization primitives.
>
> What is the actual approach to that?
> I was able to find an old mail to ntsync/winesync but sadly nothing about
> it's progress.
> Is it still in developement?

Yes, it's still in development.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #21 from Shmerl  ---
(In reply to Zeb Figura from comment #20)
> > Is it still in developement?
>
> Yes, it's still in development.

Is there any place / repo to see the development? Haven't heard any news on
this so I wonder how it's going.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #22 from Shmerl  ---
Also, is there some way to try it or it's not yet in usable state?

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #23 from Zeb Figura  ---
(In reply to Shmerl from comment #21)
> Is there any place / repo to see the development? Haven't heard any news on
> this so I wonder how it's going.

(In reply to Shmerl from comment #22)
> Also, is there some way to try it or it's not yet in usable state?

There's not really any news to report on development, but yes, it's in a usable
state. I believe the most recent repositories are:

https://repo.or.cz/wine/zf.git/shortlog/refs/heads/fastsync4

https://repo.or.cz/linux/zf.git/shortlog/refs/heads/winesync4

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #24 from Shmerl  ---
(In reply to Zeb Figura from comment #23)
> There's not really any news to report on development, but yes, it's in a
> usable state. I believe the most recent repositories are:
>
> https://repo.or.cz/wine/zf.git/shortlog/refs/heads/fastsync4
>
> https://repo.or.cz/linux/zf.git/shortlog/refs/heads/winesync4

Got it, thanks. I'll see if that can be applied to recent Wine and kernel.

Is it already in good state to propose to submit upstream or more work is
needed?

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #25 from Zeb Figura  ---
(In reply to Shmerl from comment #24)
> (In reply to Zeb Figura from comment #23)
> > There's not really any news to report on development, but yes, it's in a
> > usable state. I believe the most recent repositories are:
> >
> > https://repo.or.cz/wine/zf.git/shortlog/refs/heads/fastsync4
> >
> > https://repo.or.cz/linux/zf.git/shortlog/refs/heads/winesync4
>
> Got it, thanks. I'll see if that can be applied to recent Wine and kernel.
>
> Is it already in good state to propose to submit upstream or more work is
> needed?

Right now it's in something close to an upstreamable state. I'm currently
trying to collect performance data in order to make a good case for upstreaming
it.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #26 from Shmerl  ---
(In reply to Zeb Figura from comment #25)
> Right now it's in something close to an upstreamable state. I'm currently
> trying to collect performance data in order to make a good case for
> upstreaming it.

Sounds interesting! Can you please rebase the repos on recent kernel (like
6.2.x I guess) and Wine (8.3 I suppose)? I'd be interested in testing it /
reporting some performance comparisons to stock Wine or Wine+esync (if it
helps), but figuring out how to rebase can be complicated.

[Bug 50281] fsync patch for wine 6.0-rc1 (without staging)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50281

--- Comment #27 from Zeb Figura  ---
(In reply to Shmerl from comment #26)
> (In reply to Zeb Figura from comment #25)
> > Right now it's in something close to an upstreamable state. I'm currently
> > trying to collect performance data in order to make a good case for
> > upstreaming it.
>
> Sounds interesting! Can you please rebase the repos on recent kernel (like
> 6.2.x I guess) and Wine (8.3 I suppose)? I'd be interested in testing it /
> reporting some performance comparisons to stock Wine or Wine+esync (if it
> helps), but figuring out how to rebase can be complicated.

I'll post it if I get the time to rebase.