Summary of MSRPC / DCE/RPC interoperability / Applicability of FreeDCE for Wine (DCOM)
hi, in case you looked at the number of emails exchanged on FreeDCE and freaked out, here's a summary. if you feel that there are any relevant points missing or incorrect, please _do_ speak up. [p.s. this message not sent to samba mailing lists due to censorship decision by samba team in effect since 15th dec 2004. if anyone believes this message to be relevant to samba development please kindly forward it to the samba lists, checking first to see if you are the first person to do that. or simply feel free to add cc on any comments to any samba lists you feel are appropriate. please _do_ remember to remove my email address from any cc's to samba lists because it is likely to trigger the samba team censorship filters. thank you] 1) initial query raised: "freedce suitable for wine, esp. dcom!" 2) comments made (corrections from mark, rob and wez): "freedce doesn't have anything _like_ the capability required for DCOM - freedce's DCOM code is 'stubs'" 3) confusion resulted, clarification process began (rob, luke). conclusion: a) "freedce is almost identical to MSRPC, and is the back-end to rpcrt4.dll" b) "MSRPC as implemented in rpcrt4.dll has NOTHING to do with DCOM: DCOM compiles to code that ***UTILISES*** rpcrt4.dll" (this requires further explanation, see below). c) Porting of / Plugging in FreeDCE to rpcrt4.dll could be done "a la Apache Runtime (APR)", i.e. wrapper functions - maybe even just macros in most cases. d) extending FreeDCE to support rpcrt4.dll is easy and almost boring - even RpcImpersonateClient would be trivial to add (trivial from a Wine perspective - but see below for implications) e) lkcl suggested porting FreeDCE to use Wine's threads because dcethreads (posix draft 4 threads) have to diiiieeee. this can be done independently of Wine and there is an incentive _outside_ of Wine to _get_ it done because the dependency on posix draft 4 threads is so "last decade". f) modifying and utilising FreeDCE to understand Win32 "type string format" tables is, with the help of muddle (a program written by matthew chapman that understands exactly those tables in Win32 DLLs and EXEs) going to be a pretty trivial task. (see below for more details here) g) extension of DCE/RPC syntax to support MSRPC syntax is infinitely more straightforward than rewriting an IDL compiler from scratch. (see below) ... still with me so far? :) now here's the catch, and the catch to the catch :) the main counterargument raised to utilising freedce was not so much a technical argument, but was a resources-related and maintenance argument. 'why bother, when the approach presently being taken will "work"'? to answer that, i have to drop out of "summary" mode and into a mode of adding to and also reiterating [some of the] points raised last week. 1) it'll work - _eventually_ - and not fully [codeweavers present aims are for binary and type string interoperability only, and for only a subset of MSRPC to keep the time, development and maintenance costs down] 2) if you ever intend to aim for full interoperability with MSRPC, i recommend that you investigate how "full" pointers are implemented, because i know of a [proprietary] company that implemented that, after first only doing unique pointers, and adding "full" pointers QUADRUPLED the amount of code that they ended up with. 3) why aim to implement (from scratch) only 25% of the full solution (i.e. leaving out network interoperability by implementing your own non-NDR wire format, and not even implementing the full capabilities of the DCE/RPC IDL language) when a 95 to 98% feature-complete and free-software-license-compatible solution already exists? madness! madness, i say! 4) by utilising FreeDCE in Wine, you can kill two birds (actually three or maybe four) with one stone - two of which were aimed for with the initial DCOM work (DCOM binary interoperability at the Win32 and "type string" table levels) and two more are gained "by accident" - network interoperability and the ability to run MSRPC-based applications. 5) the effort expended so far is most certainly not "wasted" by adopting FreeDCE. if the time and effort expended so far had not been spent, how would it then be possible to appreciate the scope and scale of adapting and maintaining FreeDCE integrated to suit Wine? remember, i went through the same thing - i implemented from scratch HAND-CRAFTED marshalling and unmarshalling code in NT Domain interoperability code - almost 100,000 lines over three years - before deciding it was insane and seeking alternatives [and i found FreeDCE]. 6) if you understand the present Wine rpcrt4.dll code, FreeDCE is _so_ similar that the issue of "it's so different that codeweavers and the wine developers couldn't _possibly_ adapt to it" is just absolute nonsense and a complete non-issue: it _is_ the same code (OSF DCE 1.1 "reference" implementation) it's just that in rpcrt4.dll, DCE 1.1 has been "microsofted", and in FreeDCE, DCE 1.1 has been "gnu'd" and "autoconf'd". the question therefore, in my mind, turns on its head: "how _can_ you turn such an opportunity to save time (and money, and effort) down?" summary (well, sort of summary) ends here. the rest of this message is notes about the two. three points above. [noooobody expects the spanish inquisitionnn. the key element is surpriiise. surprise and ... the _twooo_ key elements .... ] DCOM is nothing to do with MSRPC regarding 3b) - this is where the initial confusion arose in 1). DCOM is, as rob kindly clarified, an object-orientated "thing" that uses MSRPC for its transport. it is a programming concept, not a networking API. in microsoft's c++ implementation of DCOM, once the c++ compiler has done its job, no evidence of the fact that it was DCOM remains: everything is compiled down onto MSRPC interfaces [including certain services that are required as part of DCOM]. therefore, it is both easy to confuse rpcrt4 as an implementation of "DCOM" (which it is not, it is an implementation of MSRPC) and also it is easy to forget that DCOM expects some interfaces (including the c++ APIs if you want your c++ applications to utilise DCOM) and services (e.g. the additional service in the endpoint mapper) to exist. and, according to documentation written by microsoft and available online at the open group's site, one of those services is a Win32 registry (yuk). oops :) for which there exists samba tng code that implements the MSRPC side of a Win32 registry, btw, which could easily be combined with Wine's registry to complete a full proper job. RpcImpersonateClient and ImpersonateNamedPipeClient these functions can be implemented such that FreeDCE will transfer security contexts about (to and from Win32 applications that ask for them). the process of adding the two above functions to Wine is very straightforward: pass a few blobs around, such that they can be passed to SeAccessCheck amongst other things. the _implications_ of adding them are that ReactOS must actually understand them, make use of them, etc. [otherwise, you needn't bother - just implement stubs] and _that_ means that the integration with and adoption of Samba or Samba TNG's architecture and NT Services needs to be accelerated. the Samba Team will need to be brought up to speed on how this is achieved. you may run into difficulties, there, however. their general ethos is to keep complete control over all projects that they are involved with or need to make use of (emphasis on _they_ need), and where this is not possible, they tend to reimplement from scratch any components that they require (kerberos servers, ldap servers, that sort of thing). recent developments, however, tend to suggest that this ethos is changing - but the proof of the pudding is always in the eating: their track record on cooperation with other projects is not good. you may therefore find it easier to work with the samba tng architecture for the initial implementation of these two functions, because samba tng is already split into easily manageable and understandable portions behind APIs that match or mirror the samba tng team's "best-guess" of the underlying (or unpublished) Win32 apis. and, as frequently happens, once a concept has been demonstrated as viable in samba tng, it is often adopted by the samba team - so you need not have any concerns about utilising the samba tng architecture. consider it a "proving ground". NDR "type string" interoperability - f) compared to writing an interpreter from scratch that understands these tables, adapting FreeDCE to understand MSRPC tables in DLLs and EXEs is most certainly trivial. NDR "type strings" - aka in DCE/RPC formats as "AST", is effectively a byte-code interpreted language, one in which the size of data structures and the type and offsets of members of structs is embedded, amongst almost a hundred or so other things. remember, binary-interoperability with _linux_ is totally irrelevant: you can just recompile the source code, link against static libraries, blah blah, sorted. so adapting FreeDCE for win32 "type string" interoperability it's a matter of: 1) changing some constants and alignments where necessary, to match those in MSRPC (analysing muddle and freedce) 2) improving muddle to correctly identify the data structures and their relative positions in the DLLs / EXEs - at present, muddle "guesses" some of the table's locations (eek!) and as an aside / prerequisite: 3) correctly duplicating any MSRPC extensions, including adding in support for "complex" expressioning. MSRPC interoperability - g) "interface" inheritance has already been added (by wez). "expressions" needed for size_is(complexthing) have already been added [needs more work and needs "outsourcing" to a function] range(n-n) needs to be added (trivial to do). -- -- <a href="http://lkcl.net">http://lkcl.net</a> --
Luke Kenneth Casson Leighton wrote:
hi, in case you looked at the number of emails exchanged on FreeDCE and freaked out, here's a summary. if you feel that there are any relevant points missing or incorrect, please _do_ speak up.
There's only one thing that counts for Wine, and that is patches that improve Wine. A grand scheme is nice, but it means nothing without patches. And each patch should improve something and make Wine work better in its own right. Set low goals for each patch. Having something working now is more valuable than having a chance at having something work in the future. Look forward to seeing your nice incremental improvements, submitted as bite sized patches to wine-patches. Mike
On Tue, Jan 25, 2005 at 01:29:03PM +0900, Mike McCormack wrote:
Luke Kenneth Casson Leighton wrote:
hi, in case you looked at the number of emails exchanged on FreeDCE and freaked out, here's a summary. if you feel that there are any relevant points missing or incorrect, please _do_ speak up.
There's only one thing that counts for Wine, and that is patches that improve Wine. A grand scheme is nice, but it means nothing without patches.
And each patch should improve something and make Wine work better in its own right. Set low goals for each patch. Having something working now is more valuable than having a chance at having something work in the future.
Look forward to seeing your nice incremental improvements, submitted as bite sized patches to wine-patches.
mike, hi, what is my incentive to do that? i am presenting the wine team with an opportunity to save time and effort, because i feel a sense of responsibility and a duty to do so: i hate to see people waste time and effort duplicating code - _especially_ people who do so in a free software environment. as things stand, however, the amount of effort i can spend it ends there, however, because the last time i was paid and actively encouraged to work on free software ended in 2000. i would therefore be delighted to be presented with an opportunity and an incentive to provide you with as many nice incremental improvements and bite sized patches to wine-patches as you see fit to receive, evaluate and accept - especially an opportunity that saves other people stacks of time, effort and money. l.
From: Luke Kenneth Casson Leighton
what is my incentive to do that?
Ehm, perhaps because it's fun? There must be something with opensource that makes all those volunteers want to contribute. Were you asking for funding as many times on the samba mailing lists as on the wine mailing lists? If so, I can begin to understand why they banned you, it gets _really_ annoying. Ge van Geldorp.
On Tue, Jan 25, 2005 at 10:36:10AM +0100, Ge van Geldorp wrote:
From: Luke Kenneth Casson Leighton
what is my incentive to do that?
Ehm, perhaps because it's fun? There must be something with opensource that makes all those volunteers want to contribute.
yes - that and being part of something bigger than they are. ... trouble with that is that both Wine and Samba are small parts of a much larger picture that not many people understand or appreciate. i've endeavoured to give you a glimpse of that picture.
Were you asking for funding as many times on the samba mailing lists as on the wine mailing lists? If so, I can begin to understand why they banned you, it gets _really_ annoying.
dear ge, it is not my intent to annoy. i spent nearly four years working on samba: it _was_ fun (up until 2000), and there was enough incentive in the work i was doing for security companies (ISS) and other interested parties (people who needed samba to be a PDC) to employ me to continue to research the area i was interested in. that time is now over: so i am therefore focussing on other areas where there may be interest in funding such development. i thank you for your time: when i have enough spare time and money or if someone comes forward to provide sponsorship i will provide you with all the patches in the world. l.
Luke Kenneth Casson Leighton wrote:
what is my incentive to do that?
If you have no motivation to write patches on your own time, then you're unlikely to be hired, and unlikely to be listened much either. If you write the patches, get familiar with the code and what we're trying to achieve then you'll understand the problems we're facing much better. Ideas are a dime a dozen. Patches are golden. Mike
On Tue, Jan 25, 2005 at 06:52:08PM +0900, Mike McCormack wrote:
Luke Kenneth Casson Leighton wrote:
what is my incentive to do that?
If you have no motivation to write patches on your own time, then you're unlikely to be hired, and unlikely to be listened much either.
until i have such time and money as can be spared to assist you without any guarantee of return on my time invested: so be it. i have shown you a way: i invite you to consider it. that's all i can do at this time. l.
participants (3)
-
Ge van Geldorp -
Luke Kenneth Casson Leighton -
Mike McCormack