[GSoC] Message mode pipes proposal

[Bryan Donlan]

Hi, all,
> From the GSoC project ideas page, I became interested in the proposal

for implementing message-based pipes (see bug:
http://bugs.winehq.org/show_bug.cgi?id=17195). The most recent patches
to the bug cited above appear to use a separate socketpair for each
message sent down the pipe, and must make potentially multiple
wineserver calls for each interaction with the pipe. This will clearly
begin to have problems once a large number of file descriptors are in
the air.

While a discussion on #winehackers suggested that implementing
message-based pipes efficiently and safely entirely in userspace would
be quite difficult, I feel that implementing a _correct_
implementation (regardless of efficiency) would be a practical goal.
After creating a correct implementation, work could then be done to
improve performance.

First, a bit about me: I am a computer science student at the
University of Massachusetts, heading into my senior year. I have had a
lot of experience with Linux development, somewhat less on the Windows
side. I am fluent in English and Japanese, although I doubt Japanese
language experience would be helpful for this project :)

In previous summer internships I have developed code for interprocess
communication; in one instance, I wrote a ring buffer synchronization
protocol to transfer data from a Windows user-space program to a
real-time process running in kernel-mode (hosted via a third-party
Windows real-time HAL).

I propose the following overall plan:
* First, I will prepare a comprehensive test suite for message-based
pipes. Benchmarks would also be prepared to help determine whether
optimization is necessary. This will likely borrow from the tests
already posted to the bug I linked above.
* I will write a wineserver-internal implementation of message-based
pipes. That is, all NtReadFile/NtWriteFile() requests would be
completely redirected to wineserver; there would be no attempt to
expose a file descriptor for the client process to access directly.
Wineserver in turn would simply manage a simple, in-memory queue of
messages. Although this is unscalable and has a lot of context switch
and copying overhead, it is simple, and would allow programs making
light usage of message-based pipes to work. It also provides a clear
place to hook on a new, faster protocol that requires a custom
NtReadFile/NtWriteFile.

After this point, I will begin work on optimizing the implementation.
Some ideas on how to do this might include:
* Implement the pipe using a ring buffer in shared memory. Client
processes can then directly access the pipe buffer to pass data
without wineserver's involvement. This has the downside that a
user-mode process can inadvertently corrupt the pipe's state; this may
be acceptable if the effects can be limited to a trashed pipe buffer,
instead of crashing unrelated processes.
* Implement the pipe using a ring buffer in shared memory, but expose
only a read-only file descriptor to client processes. This avoids the
corruption issues, but writes must be managed by the server and will
incur overhead.
* Implement the pipe as a single server-managed socketpair over which
shared memory backing files are passed with SCM_RIGHTS, plus a lock
(possibly implemented in shared memory using futexes, or by flock on a
/dev/shm file). Clients acquire the lock, then MSG_PEEK the socketpair
to retrieve an anonymous (unlinked immediately after creation) shared
memory file descriptor. The shared memory buffer holds a count of
remaining bytes, plus the actual message data. Clients then perform a
normal recvmsg() to dequeue the message if they successfully retrieved
the entire message. This approach is complex, and it is unclear what
the performance impacts of having so many files being thrown around
would be. However, it does avoid the context-switch costs and
wineserver load that are a problem in the implementations currently on
the bug tracker.
* Implement a Linux kernel module to implement message-based pipe
semantics natively. Although quite efficient, this will only apply to
Linux, and may introduce difficulty keeping up with Linux kernel
changes in the future. It is also unclear whether distributions and/or
upstream Linux maintainers will welcome this approach.

I will likely end up trying out several of these approaches and
comparing actual performance results.

Finally, if time allows, I will also investigate integration with
samba, in order to support connecting to named pipes on remote
servers.

I would appreciate any comments on this proposal prior to actual submission.

Thanks,

Bryan Donlan