Hello!
PE signing is already publicly documented by Microsoft, and we have an open-source tool (osslsigncode) capable of signing PE and Cabinet. A signed MSI dump easily reveals that the signature is under the '\005DigitalSignature' stream, and it's a regular pkcs#7 signature, whose content info type is 1.3.6.1.4.1.311.2.1.30 (SPC_SIPINFO_OBJID whatever that might be). To ease debugging, I added asn1parse of signature to msitools with "msidump --signature". You can also compare two MSI signatures with msidiff (http://git.gnome.org/browse/msitools)
I found this blog http://qistoph.blogspot.com.es/2012/01/manual-verify-pkcs7-signed-data-with.... very helpful, so my testing setup currently is: - self signed pkcs12 certificate - minimal "starting point" MSI crafted with msibuild mini.msi -s '', modified with hex editor for 0 length streams - signtool from 7.0A sdk, used simply "sign /f cert.pfx /v /p passphrase mini.msi"
From there, I get a "canonical" message digest sha1 value of
98C090F33284468D64F30EE7B8C00894135D54C8. Only the stream content seems to be used to compute it. All modifications I can think of to the MSI do not seem to alter that sha1, none of: OLECF internal data, stream name, number of (empty) streams, total file length, position of the streams, or whatever I could tweak in the OLECF data seems to change the resulting sha1. So it very much looks like an initial "seed", somehow?
Based on my limited knowledge of sha1 computation, I have tried initializing the sha1 by modifying its initial value manually (the h values) to match with the "canonical" sha1, and added 1 byte to a stream. I was hoping to get the same result, but that didn't work that way. It could be that my modified sha1 is wrong, or that they apply further transformation on the stream content before computing the checksum, although this would be surprising since they don't do that for PE or cabinet file.
Any suggestion or help is welcome!