Re: [PATCH v11 0/3] MR741: ntdll: Implement NtFlushProcessWriteBuffers.

[Jinoh Kang (＠iamahuman)]

Jinoh Kang (@iamahuman) commented about dlls/ntdll/unix/virtual.c:
> -
>    if (mem == MAP_FAILED)
> -
>        goto failed;
> -
>    /* If the memory is locked, e.g. by a call to mlockall(MCL_FUTURE), the madvise() call below
> -
>       will fail with error EINVAL, so unlock it here */
> -
>    ret = munlock( mem, page_size );
> -
>    /* munlock() may fail on old kernels if we don't have sufficient permissions, but that is not
> -
>       a problem since in that case we didn't have permission to lock the memory either */
> -
>    if (ret && errno != EPERM)
> -
>        goto failed;
> -
>    dontneed_page = mem;
> - }
> - /* Force the page into memory to make madvise() have real work to do */
> - *mem = 3;
> - /* Evict the page from memory to force the kernel to send an IPI to all threads of this process,
> -
>   which has the side effect of executing a memory barrier in those threads */
> - success = !madvise( mem, page_size, MADV_DONTNEED );

It turns out that `MADV_DONTNEED` semantics is [not portable]. [For example, it doesn't necessarily trigger TLB shootdown immediately in native FreeBSD][freebsd_dontneed]. I'm not sure ScyllaDB has first-class support for non-Linux OSes, either.

I suggest reverting to mprotect(), as documented in the paper I cited earlier in this MR discussion.

[not portable]: https://www.man7.org/linux/man-pages/man2/madvise.2.html#DESCRIPTION
[freebsd_dontneed]: https://github.com/freebsd/freebsd-src/blob/23d4d0fcc1be3d2f44054dd12725098a...