i've been updating this page with relevant information that is part requirements specification, part documentation. of particular relevance is the lack of support (in both tng and samba 3) for transferring SetNamedPipeHandleState semantics, although the "stub" functionality inside tng (smbd/pipe-proxy.c) at least records the NP message-mode flags, and then _completely_ ignores them ha ha :)
exactly how this is to be implemented is unclear, and to be absolutely honest, by far the simplest-sounding approach may be to actually compile samba up as a win32 app!
under such circumstances, it would be incredibly easy to then call the wine-emulated NamedPipe win32 functions CreateNamedPipe, SetNamedPipeHandleState etc. etc.
given that samba tng has _already_ successfully been ported to win32, it has a head start here. and in that regard, ReactOS has it made, done and dusted!
the alternative is... gaahd, i dunno... messy, to say the least, and *shudder* an mmap'd tdb is sounding _really_ attractive.
tell me if this sounds reasonable. basically, a mini-wineserver protocol - a mini-SMB protocol - is required. commands to "send", "receive", "setnamedpipestate", "waitnamedpipestate" and "close". all required.
* samba 3 / tng create or access named pipes as emulations over unix domain sockets, including sending a security context.
in samba 3, that's called a SamInfo3, but it looks suspiciously like a NETLOGON NET_USER_INFO_3) and in tng it's a NET_USER_INFO_3... so... yep, it looks like just the name has been changed.
* a program wants to create or access a named pipe; it asks wineserver (via a new function) to create or open up a unix domain socket.
the locations will be the same. i.e. tng / samba3's unix domain socket pipe locations same as wineserver's.
* sending / receiving of the security context blob is done in the dlls NOT in wineserver, so as to avoid blocking wineserver.
* read, write, snphs, wnphs are all done with a 2-byte command prefix, followed by parameters, followed by data. exactly the same as is done in SMB and in wineserver.
... you know... if SMB wasn't so damn complicated i'd say bugger it, just send the SMBs to Wine, for _it_ to sort out!! in fact, with cliffsd's auto-generated SMB parsing code, that might even still be a valid option, given that the relevant code for parsing the SMBs is public domain.
what do people think?
does this sound like a reasonable proposition - a mini SMB/wineserver protocol - for use for inter-communication between wine and samba in order to exchange named pipe traffic?
l.
Hi Luke,
does this sound like a reasonable proposition - a mini SMB/wineserver protocol - for use for inter-communication between wine and samba in order to exchange named pipe traffic?
Well, I think we have to crawl before we can walk. I think it's more important that we have a working local named pipe implementation before we start thinking about SMBs from Wine to a remote machine. Usually, having a specific app in mind really helps drive an implementation, too. So, for instance, local message-mode named pipes will help Python run in Wine: cool. There are of course many apps that might want to do SMBs over named pipes.. but having a specific one we're trying to fix, along with someone motivated to get it working, helps. --Juan
On Sun, Feb 15, 2009 at 9:43 PM, Juan Lang juan.lang@gmail.com wrote:
Hi Luke,
does this sound like a reasonable proposition - a mini SMB/wineserver protocol - for use for inter-communication between wine and samba in order to exchange named pipe traffic?
Well, I think we have to crawl before we can walk.
:) that's why i said "mini" smb / wineserver protocol, comprising:
[2-byte command] auth, waitnphs, setnphs, read, write [parameters] [variable-data]
which is incredibly incredibly simple, and would take ... what... 200 lines of code, to write?
setting the "auth" data blob to NULL _is_ "crawl before walk".
I think it's more important that we have a working local named pipe implementation before we start thinking about SMBs from Wine to a remote machine.
follow the chain, here... :) not least, to make sure it sounds reasonable!
well, the more i thought about how awful the "multiple socket-pair queue" design is, and the more i think about what's on the MSDN "CreateNamedPipe" page, the more i believe that a per-file-handle Mutex is utterly utterly essential.
and if you're going to do that [per-file-handle mutex], then you _can_ use that to emulate blocking-read semantics correctly instead of read(), select(), poll() etc.
_that_ means that you can do whatever you like - send headers down the socket, send lengths down the socket, do whatever-you-like-down-the-socket because there will be only one reader.
_that_ means you can get away with sending quote commands quotes.
the combination of sending "commands" and exclusivity means you don't have to have the multiple socketpair queue
and, while you're at it, it so happens that you can replace the socketpair() with a sockaddr_un pair (hooray!)
so, you get to write a simplified design that meets the requirements _and_ is well on the way to meeting the other requirements - being able to interoperate with samba.
Usually, having a specific app in mind really helps drive an implementation, too.
i like to think ahead. saves a lot of time and effort in the long run.
l.
On Sunday 15 February 2009 19:47:13 Luke Kenneth Casson Leighton wrote:
i've been updating this page with relevant information that is part requirements specification, part documentation. of particular relevance is the lack of support (in both tng and samba 3) for transferring SetNamedPipeHandleState semantics, although the "stub" functionality inside tng (smbd/pipe-proxy.c) at least records the NP message-mode flags, and then _completely_ ignores them ha ha :)
exactly how this is to be implemented is unclear, and to be absolutely honest, by far the simplest-sounding approach may be to actually compile samba up as a win32 app!
That'd force you to run Wine as root (or using POSIX capabilities) to be able to bind to < 1024 ports. Also, if the box is also running Samba, we're at the old clash of who owns ports 137-139 and 445.
given that samba tng has _already_ successfully been ported to win32, it has a head start here. and in that regard, ReactOS has it made, done and dusted!
Yes, but ReactOS doesn't run any other smbd, while I'd argue that's relatively common on Unix machines these days.
the alternative is... gaahd, i dunno... messy, to say the least, and *shudder* an mmap'd tdb is sounding _really_ attractive.
I'm not sure why you'd want to make assumptions about the database backend the samba server is using. Assuming sometime this year we actually manage to roll out franky as a release, a bunch of databases will end up being LDB, not TDB.
tell me if this sounds reasonable. basically, a mini-wineserver protocol - a mini-SMB protocol - is required. commands to "send", "receive", "setnamedpipestate", "waitnamedpipestate" and "close". all required.
[...]
does this sound like a reasonable proposition - a mini SMB/wineserver protocol - for use for inter-communication between wine and samba in order to exchange named pipe traffic?
Is there any reason we can't use plain old RPC for talking to Samba? We need to authenticate somehow, of course. There would be long term advantages of setting up a more generic pipeline to talk to Samba, though. I'm thinking of browsing support, netapi support, and a couple of other things.
Cheers, Kai
On Mon, Feb 16, 2009 at 8:22 AM, Kai Blin kai.blin@gmail.com wrote:
On Sunday 15 February 2009 19:47:13 Luke Kenneth Casson Leighton wrote:
i've been updating this page with relevant information that is part requirements specification, part documentation. of particular relevance is the lack of support (in both tng and samba 3) for transferring SetNamedPipeHandleState semantics, although the "stub" functionality inside tng (smbd/pipe-proxy.c) at least records the NP message-mode flags, and then _completely_ ignores them ha ha :)
exactly how this is to be implemented is unclear, and to be absolutely honest, by far the simplest-sounding approach may be to actually compile samba up as a win32 app!
That'd force you to run Wine as root (or using POSIX capabilities) to be able to bind to < 1024 ports.
strictly speaking, just wineserver.
and, in order to secure the unix domain sockets, given that you're sending a security context and effectively telling the listener to "seteuid" to that security context (both in NT terminology and also ultimately _really_ telling listeners to do seteuid once you've converted the NT SID to uid,gid using the winbindd built-in SURS mechanism) *deep breath :) *
in order to secure the unix domain sockets, you have to run wineserver as root, anyway.
side-note: _but_ (spelling this out for the benefit of juan and other wine developers), you only need to do that [run wineserver as root] if you actually start doing nt security contexts. so, if you start off with a named-pipes sockaddr_un emulation design that is neeaarly interoperable with samba, then people can at least experiment, test it, privately run wineserver as root, but public releases always _never_ send that security context etc. etc.
Also, if the box is also running Samba, we're at the old clash of who owns ports 137-139 and 445.
yehh, i knoww.
given that samba tng has _already_ successfully been ported to win32, it has a head start here. and in that regard, ReactOS has it made, done and dusted!
Yes, but ReactOS doesn't run any other smbd, while I'd argue that's relatively common on Unix machines these days.
*thinks*... ok, so.. dropping the "but", the sentence makes more sense to my literal-minded brain: you're saying that because reactos doesn't have posix compliance, and so no smbd to speak of, it's in a better position to choose. which leaves its users in an unfavourable position.
the best thing there would be to give steven's samba4 Win32 port a bit higher priority, yes?
the alternative is... gaahd, i dunno... messy, to say the least, and *shudder* an mmap'd tdb is sounding _really_ attractive.
I'm not sure why you'd want to make assumptions about the database backend the samba server is using.
ah you misunderstand .... *thinks a bit more*.... where are you coming from... ok i get it.
answer 1: if you have the six namedpipe functions rolled out into a dynamically loadable module (same as what you call VFS layers) then it's up to the dso to choose the implementation of namedpipes. so samba3 / samba 4 uses sockaddr_un, with its own unique dedicated dso, *shrug* you're a user / sysadmin, you want to interoperate with wine? change some config files, replace one component - the namedpipes-emulation-dso, bang, you're done.
and in that wine-samba npemu dso, it _happens_ to use an mmap'd tdb, or it _happens_ to use anything-it-likes, and that has _nothing_ to do with samba's internal use or otherwise of tdb, yes?
answer 2: i like the sockaddr_un idea much better than an mmap'd tdb anyway :)
tell me if this sounds reasonable. basically, a mini-wineserver protocol - a mini-SMB protocol - is required. commands to "send", "receive", "setnamedpipestate", "waitnamedpipestate" and "close". all required.
[...]
does this sound like a reasonable proposition - a mini SMB/wineserver protocol - for use for inter-communication between wine and samba in order to exchange named pipe traffic?
Is there any reason we can't use plain old RPC for talking to Samba?
i can think of several, they're not exactly "absolutely you can't" reasons, more of "yukk" reasons
reasons _for_ the idea of using RPC:
1) writing an IDL file npemul.idl would be _great_!
int open_npemp([in] NET_USER_INFO_3 *ctx, [in] UNICODE_STRING *pipe_name, [in] int messagemode, ..., [out, handle] *pipehnd); int set_npemuhs([in, handle] *pipehnd, int new_messagemode, ...); int write_npemu([in, handle] *pipehnd, [in, out] *length, [in] LPVOID buffer, ...);
exactly as CreateNamedPipe, NtReadFile, NtWriteFile and friends
thanks to WIDL.EXE, dceidl and pidl, this is much much simpler; anyone can get the idl file and "interoperate".
reasons against:
1) why drag rpcrt4 into the mix when you can do about 150 lines of code to do the job (ok, excluding NET_USER_INFO_3 marshalling / unmarshalling!) - it's only what.. 5 commands (auth, snphs, wnphs, read, write) so is utterly trivial to write.
2) forcing RPC authentication over unix domain sockets is what... a _five_ round-trip exchange? involving contacting potentially a remote BDC? just to open up a named pipe?? when all you _actually_ want to do is pass over a pre-existing security context that you should _already_ have? ... naaah - leeave it aaaht, guvnor :)
this was the reason why i added the "Pre-Authentication" PDU which, btw, you really _do_ have to have the NetBIOS names and Domain Name as well as the NET_USER_INFO_3 structure, because it's a requirement / part of the authentication information.
remember, you don't just have _one_ MSRPC service, as part of NT Domains authentication, you have at _least_ three involved, with recursive calls inside NETLOGON off to other machines, which then start the multi-MSRPC circus _again_! you want to add PDU negotiation plus a 3-way AUTH to each and every single one of those go ahead :)
3) conceptually, dragging DCE/RPC into the mix, to proxy traffic which has nothing to do with DCE/RPC, so that in most Wine instances it's just a massive overhead, and then by _pure_ coincidence it just so happens that _some_ uses will end up being ncacn_np, so you'll get DCE/RPC over DCE/RPC over sockaddr_un and i think not only will that confuse the crap out of people but also it risks getting into an infinite recursive loop unless you are reaaalllly careful.
overall, then, whilst it sounds on the face of it like a damn good idea at the "high level", at least in my mind the odds come out in favour of a trivial custom protocol, following the lead of the wineserver protocol.
We need to authenticate somehow, of course.
yes and no. initially, i'd say "stuff it - have a #ifdef DISABLE_SECURITY" switch for special developer-only builds. this is what happened for TNG, so that the embedded systems guys could cut out vast amounts of code.
yes, because yes you _will_ need to drag in that "initial security context" from somewhere. this can be done in a couple of ways:
1) talk to winbindd. if you're logged in via a system that uses winbindd, you already _have_ an nt security context, and winbindd is looking after it. grab it, cache it, shove it over the npemu when needed.
2) present a "user, pass, domain" logon box to users - a la GINA (hooray!) talk NETLOGON auth (code exists in winbindd, port it to MSRPC or just treat it as a library; actually that code came from TNG so you could grab that instead. again). NETLOGON auth gives you a security context. grab it, cache it, shove it over npemu when needed.
3) present a "user, pass, domain" logon box to users, and use PAULA Daemon - this is an OpenGroup RFC defining how UNIX systems should "hook in" to NT Domains Authentication. it's a DCE/RPC jobbie, and the IDL file is included in the PAULA specification. a dedicated MSRPC service (which will run as a samba-based service) performs the actual NETLOGON auth, thus vastly simplifying the whole deal especially from wine's perspective.
http://www.opengroup.org/comsource/techref2/NCH1222X.HTM
... *re-reads*... actually, that's far too simplified - and is missing the NET_USER_INFO_3 structure, which is essential. so - a _modified_ PAULA protocol then :)
There would be long term advantages of setting up a more generic pipeline to talk to Samba, though. I'm thinking of browsing support, netapi support, and a couple of other things.
yehhh, i'd be inclined, there, to do each of them on a case-by-case basis.
if you're thinking of NetShareEnum, NetServerEnum, NetServerGetInfo, NetUserGetInfo and friends, those are \PIPE\LANMAN as you know, and i _believe_ you _might_ be able to get away with putting that over NamedPipes, creating a separate "lanmand" service.
if you look at cliffsd you'll find that i cleanly auto-generated the LANMAN code by actually taking the LANMAN specification document and using it as the IDL file !! :)
the reason i mention this is because i ended up being able to auto-generate client-side functions _as well_ as a server-side stub [which i then expanded out, cleanly, in a format that will be much more familiar to Wine developers than to Samba developers].
and the important point is: it was all as a separate service, with smbd performing proxying into lanmand.
i _think_. :)
it was a bit of a long time ago and i only spent a couple of months on it. andrew tried to keep me busy / out of the way, and i think he was a bit shocked at how much progress i made in a short period of time - an operational smb server with a clean design and cleanly-auto-generated client libraries.
which didn't help. never mind.
anyway.
so yeah, there's code there that can be used in a couple of ways:
1) the client-side cliffsd code could be used to actually call in, to smbd, to obtain quote real quotes answers, because the client-side functions are network-capable versions of NetShareEnum, NetServerGetInfo etc. etc. all auto-generated and all _should_ be pretty much identical to the Win32 API, near-as-damnit.
2) the server-side cliffsd (lanmand) code could have its smbd-related guts thrown away and instead replaced with wine-related gubbins - calls to wine's "real" NetShareEnum, "real" NetServerGetInfo etc. functions.
3) some sort of weird hybrid.
4) other
but, at least the cliffsd code opens up some options that would otherwise be really quite big projects.
ok, enough. gotta do some work.
l.
ok, written up as a specification - http://lkcl.net/namedpipes/namedpipes-emulation.txt: please review and provide feedback. kai, volker, you may wish to forward this to the samba technical lists. l.
-
Juan Lang -
Kai Blin -
Luke Kenneth Casson Leighton