see implementation in dual BSD-licensed _and_ LGPL-licensed freedce reference implementation, on which MSRPC was originally blatantly copied (thanks to the BSD license on the 1.0 reference implementation).
e.g. online annotated copy here: http://www.sfr-fresh.com/linux/misc/freedce-1.1.0.7.tar.gz:a/freedce-1.1.0.7...
line 2470.
anyone who is doing wine dce/rpc development _really_ needs to be looking at the reference implementation, side-by-side, line-for-line, in virtually every aspect except for exception handling, threading, binary-interoperability with the typelibraries and a few other areas.
l.
come on folks, pick up the ball. acknowledge that you're aware that there is an authoritative source of information on this horrendously-complex topic, in the form of nothing less than a fully functioning reference implementation.
so this:
Ok, I'll try to find out. Rob, could you suggest a good documentation source on RPC? I'm not experienced in it.
Sure, although this particular aspect isn't well documented.
simply isn't true.
it's _authoritatively_ documented, so concretely documented you could break it into a million pieces to sink a million people who walked into your turf, if you were in da mafia or summink :)
so - carrying on from where i hinted at you should investigate, the "real" work is done here:
/* * Call the protocol service to do the real work of creating the * socket(s), setting them up right, and adding them (via calls * to rpc__network_add_desc). */ (*net_epv->network_use_protseq) (pseq_id, max_calls, rpc_addr, endpoint, status);
so where is that function? well, it's not _a_ function, because net_epv is a pointer to a vector table [array of higher-order-functions]. so the design of freedce is _incredibly_ well done, and allows for new transports to be dropped in at will. and new authentication schemes. etc.
so, where is _an_ example of a function that's pointed to by network_use_protseq?
a quick grep through the code turns up this:
ncklib/transport/cn/cninit.c: rpc__cn_network_use_protseq,
having a quick look at that file actually shows _two_ connection-based (cn) protseqs - one is tcp and the other is named pipes (which i'd added in 2005). both of those are in cninit.c
to answer the specific question about allocation (NULL or not NULL) you have to chunder through quite a bit of convoluted lookups to find out where _this_ goes to:
(*naf_epv->naf_addr_alloc) (
_yes_ as you can see, NULL is an acceptable endpoint parameter and rob is correct in that it means "please allocate me aaabsolutely aanything you like, per transport".
you can tell this from what happens inside the code. in TCP/IP terms, NULL means "give me an socket on 0.0.0.0 with _any_ port number". same in UDP terms. in ncacn_np terms, i don't honestly know - you'll have to do some experiments to check - but you have two choices:
1) error 2) a random pipe name e.g. \pipe\0x00000NNN.0x0000NNNN
grep naf_addr_alloc shows one file - ncklib/include/comnaf.h which shows promise, showing you what you want, but the trail goes dead there, until you actually _look_ at comnaf.h and you can see these:
/* * Declarations of the Network Address Family Extension Service init * routines. */ PRIVATE void rpc__unix_init _DCE_PROTOTYPE_ (( rpc_naf_epv_p_t * /*naf_epv*/ , unsigned32 * /*status*/ ));
PRIVATE void rpc__ip_init _DCE_PROTOTYPE_ (( rpc_naf_epv_p_t * /*naf_epv*/ , unsigned32 * /*status*/ ));
PRIVATE void rpc__dnet_init _DCE_PROTOTYPE_ (( rpc_naf_epv_p_t * /*naf_epv*/ , unsigned32 * /*status*/ ));
PRIVATE void rpc__osi_init _DCE_PROTOTYPE_ (( rpc_naf_epv_p_t * /*naf_epv*/ , unsigned32 * /*status*/ ));
which are for unix domain sockets (which i think i added as an extra NAF in 2004), TCP/IP and UDP/IP, DecNet and goodness knows what OSI is i _so_ don't want to know :)
_again_ it's hard to track down rpc__ip_init() and friends but jumping straight to ls ncklib/naf/*.c (because duh the header file ncklib/include/comnaf.h is the clue) you can see e.g. ipnaf.c
hooraaay, there's the vector table, rpc_naf_epv_t, with the static function addr_alloc as the first member, so in the IP cases, TCP/IP and UDP/IP, addr_alloc in ipnaf.c is what's called by that (*naf_epv->naf_addr_alloc) () call.
addr_alloc calls addr_set_endpoint. addr_set_endpoint does this:
if (endpoint == NULL || strlen ((char *) endpoint) == 0) { ip_addr->sa.sin_port = 0; *status = rpc_s_ok; return; }
so - hooray! you now _authoritatively_ have your answer, thanks to the reference implementation.
when NULL is used as the endpoint, you're expected to be given _any_ available address (which in IP terms means "any port").
now you have to do the same thing for ncacn_np. a random and globally unique pipe name will be required.
remember also do do exactly what the reference implementation says, when the endpoint _isn't_ NULL, and strip off leading slashes.
l.
On Sat, Feb 7, 2009 at 4:09 PM, Luke Kenneth Casson Leighton lkcl@lkcl.net wrote:
see implementation in dual BSD-licensed _and_ LGPL-licensed freedce reference implementation, on which MSRPC was originally blatantly copied (thanks to the BSD license on the 1.0 reference implementation).
e.g. online annotated copy here: http://www.sfr-fresh.com/linux/misc/freedce-1.1.0.7.tar.gz:a/freedce-1.1.0.7...
line 2470.
anyone who is doing wine dce/rpc development _really_ needs to be looking at the reference implementation, side-by-side, line-for-line, in virtually every aspect except for exception handling, threading, binary-interoperability with the typelibraries and a few other areas.
l.
2009/2/22 Luke Kenneth Casson Leighton lkcl@lkcl.net:
come on folks, pick up the ball. Â acknowledge that you're aware that there is an authoritative source of information on this horrendously-complex topic, in the form of nothing less than a fully functioning reference implementation.
I don't think anyone questions this. But (speaking as an observer, not someone with any particular power to garner interest for this), I suspect that your best bet for getting traction on this in Wine is:
* some apps that use this and work in Windows, but won't work on Wine without the admittedly bletcherous interface being implemented - that is, a good reason to implement this particular interface
(remembering that there are large chunks of win32 that Wine doesn't bother implementing because it turns out they're all but unused by actual app developers)
* then a conformance test or several that works in Windows but not in Wine.
That is, the first thing is a real-life use case or several.
- d.
David Gerard-2 wrote:
2009/2/22 Luke Kenneth Casson Leighton lkcl@lkcl.net:
I don't think anyone questions this. But (speaking as an observer, not someone with any particular power to garner interest for this), I suspect that your best bet for getting traction on this in Wine is:
- some apps that use this and work in Windows, but won't work on Wine
without the admittedly bletcherous interface being implemented - that is, a good reason to implement this particular interface
(remembering that there are large chunks of win32 that Wine doesn't bother implementing because it turns out they're all but unused by actual app developers)
- then a conformance test or several that works in Windows but not in
Wine.
That is, the first thing is a real-life use case or several.
hiya david, thanks for responding. it's going to be difficult for me to explain this, but i'm going to try. part of the difficulty comes from being able to look at the two sets of code and see very clearly that the wine rpcrt4 implementation is line-for-line virtually identical - it _has_ to be, otherwise there will be no interoperability.
under ordinary circumstances, with something as relatively simple as a publicly-document MSDN function call, one where you can pretty much go out on the internet and find working examples, i'd say - yep, great: grab some of that, knock up a test case, go for it.
however, MSRPC isn't part of the "public API" - it's as if you'd expect microsoft to document the internal workings of their [proprietary] c / c++ compiler, because WIDL.EXE is a "tool" not an API. it was so fundamentally ingrained into the bedrock that it was - and still is - quietly taken for granted.
so there _is_ no documentation, there are no test cases or examples on the internet to copy: you create an IDL file, you hand it to WIDL.EXE, WIDL.EXE and the endpoint mapper and rpcrt4.dll all do their jobs, and you don't get a squeak out of the "documentation" from microsoft - because there isn't any.
the point is: MSRPC is "core" technology. seamless, transparent, quietly-doing-its-job.
and it's so forgotten-about, even inside microsoft, that even microsoft's own engineers make the daft mistake of putting [in,out] onto buffers that can go as large as 10mb in size, resulting in 8mb of uninitialised crap being sent by the spoolss service, an ERROR_MORE_DATA along with the same 8mb uninitialised crap being returned and a new suggestion to send 8.1mb of crap, which the API dutifully obeys... :)
it's probably increased significantly, but last time i checked (2005?), wine's source code came in at a whopping one million lines. the DCE/RPC 1.1 reference implementation, the foundation on which MSRPC was created with some (relatively) minor and strategically important enhancements, comes in at 250,000 lines.
MSRPC isn't just some "optional bit of the Win32 API" - it's a bit more fundamental than that. the following are all MSRPC services:
* The Registry Service (WINREG) * The Service Control Manager (SVCCTL) * Printing (SPOOLSS) * WKSSVC (NetWkstaGetInfo and friends) * SRVSVC (NetShareEnum, NetServerGetInfo and friends) * LSARPC (LsaQueryInformationPolicy and friends) * SAMR (very _very_ indirectly - NetUserGetInfo and friends) * NETLOGON (very _very_ indirectly the GINAs talk to this. it's complicated...)
i could go on - but you should be getting a clear picture that a very very large amount of the quotes public quotes Win32 API is fundamentally based on something as quietly unobtrusive as an extremely effective RPC mechanism.
oh - and then there's DCOM, on top of which you have MSHTML, MS-XML, and much much more. i looked last week and there are about _thirty_ applications in Wine that import stdole.tlb (a good indication that they are DCOM-based services).
so it's not like we're talking about a "little function or two" - we're talking a quarter of a million lines of code in order to get full and total interoperability, and that's _without_ having the NT Domains Authentication infrastructure, which in samba tng comprises about another ... 80,000 lines of code for an LSA, Samr and NETLOGON implementation, along-side 200,000 lines of code for an actual SMB server, without which you cannot get a NamedPipes transport (over SMB Trans2 IPC$).
so it would seem quite logical, against that background and context, to pull in all the extra help that you can get, otherwise you're quite likely to be yet _another_ decade before the job's completed. which is why i find it so frustrating when people are unresponsive when i point people at the fully-functional reference implementation, or they go "duuuuh? y'whaaat?" :)
not to knock the progress that's been made so far on WIDL.EXE and rpcrt4, it's absolutely fantastic to see. however, you haven't got round to adding in any authentication plugins, yet, nor hooked in to an SMB server (to get NamedPipes) and so much more, which, when that's done, takes Wine up to a wholllle new level.
so, david, i understand that it seems logical to create a few test cases to identify missing functionality, but we're talking about a sub-project that's 15 to 20% of the size of the codebase of Wine, and is Wine's fundamental underpinning infrastructure, and has no available documentational equivalent in the MSDN.
the best guide you can get (if you can still find it) is a KB article from around ... 1996 or earlier which, when DCE/RPC was popular in Government projects, helped you to port DCE/RPC applications to MSRPC. it had a header file which provided function-name and data-structure mappings from dce_rpc_function_names to MsRpcFunctionNames e.g. #define rpc_server_use_protseq_ep RpcServerUseProtSeqEp and there were well over 200 of these conversion #defines. they only helped you so far because the prototype in dce/rpc is this:
PUBLIC void rpc_server_use_protseq_ep (rpc_protseq, max_calls, endpoint, status)
unsigned_char_p_t rpc_protseq; unsigned32 max_calls; unsigned_char_p_t endpoint; unsigned32 *status;
and the prototype for MSRPC is this:
RPC_STATUS RPC_ENTRY RpcServerUseProtseqEp( unsigned char *Protseq, unsigned int MaxCalls, unsigned char *Endpoint, void *SecurityDescriptor );
note the addition of the security descriptor, but also note the moving of the "status code" as a return result in the MSRPC version and as a parameter in the DCE/RPC version? bit of a bugger, that one, when you have 250,000 lines of code to wade through :)
not only that, but as you can see from the subject line of this discussion, MSRPC adds "dual" functions - both ascii (A suffix) and UCS-2 versions (W suffix) of the _same_ function, with the ascii version typically doing a dumb-conversion to UCS-2 and then freeing up the strings created to do it.
so - to reiterate, and answer your question: MSRPC isn't just "a few public documented APIs" - it's more fundamental than that. and the core functionality of wine critically depends on having MSRPC working correctly and working well, and, as you can see, it's a big damn job.
when i saw your message i did that thing, you know, where you put your hand over your eyes and pull down, to stretch your face? :) it was like, "ohh maaan - how the _heck_ am i going to explain this to david? where do i _start_?" :)
hope this clarifies things.
l.
-
David Gerard -
lkcl -
Luke Kenneth Casson Leighton