MSRPC / FreeDCE interoperability of non-encapsulated onions.

[Luke Kenneth Casson Leighton]

On Thu, Jan 27, 2005 at 10:38:41AM +0000, Luke Kenneth Casson Leighton wrote:
> > Also, I don't believe your other comment about switch_is is correct. AFAIK
> > MIDL does use the size of the member / parameter used in switch_is for
> > the descriminant size (although I'm not sure how an expression within
> > switch_is would affect the size). Perhaps switch_type was also used?
>
> DCE/RPC idl absolutely _must_ have a switch_type explicitly
>  specified and yes, a switch_type of "long" was used.
>
> ironically, MIDL truncates that to a short when INTERNALLY doing case
>  statements to determine what structure member is being decoded in the
>  non-encapsulated union.
>
> yet _more_ ironically, there is evidence that it STILL allows the
>  full 32-bits to be marshalled up to the caller - thus presenting
>  the users with a 32-bit switch_is parameter, the top 16-bits of which
>  is total garbage off of some uninitialised area of memory.
>
> oops!

okay - it's confirmed.  i'm cc'ing this to the wine-devel
 list because they too are implementing MSRPC, and whilst it
 may not appear to be relevant for on-wire interoperability
 (which they are _not_ working towards in their implementation
 of rpcrt4.dll) it _may_ turn out to be relevant for application
 interoperability.
 
 michael, if you're in contact with the samba team, perhaps you
 could relay this to them because they have set up some fascist
 censorship where i cannot post to any addresses at samba.org.

 okay.
 
 you're not going to believe it, or like it, but i can only
 guess it must be something to do with "default" behaviour
 that microsoft added to MIDL.

 *sigh*.

 what i am _guessing_ is that microsoft considered it silly to force
 people to specify switch_type on parameters, so i'm _guessing_ that
 they allowed a "default" of a 16-bit value (short).

 ... and then forgot to automatically "increase" that to 32-bit
 if the switch_is on a non-encapsulated union was actually a long.

 consequently, ha ha, there are circumstances under which
 the first 16 bits of a 32-bit switch value is placed on-wire
 just before the non-encapsulated union is, and the remaining
 16 bits is filled with garbage due to alignment of the
 non-encapsulated union.

 _but_, the data structure _containing_ the item using the switch
 is of course marshalled correctly: the switch item receives the full
 32-bits, 16 of which is garbage.

 consequently, applications may _expect_ that garbage to be
 there - certainly the application that i am exploring will
 fail if you don't notify it of the full 32-bits of the switch
 value - including the garbage - in further function calls (!)

 yet when i do pretty-printing of the data structures received off-of-wire,
 i must still typecast the 32-bits to 16 in order to do my switch statement!!

 there aren't any case statements of 0xffef001e _that's_ for sure, it's
 actually 0x1e which is a STRING.

 god, i love microsoft interoperability, it's such a hoot.

 l.