[Bug 4635] Photoshop 7 fails to initialize Vbox!(0002-0073-0066)

[Wine Bugs]

http://bugs.winehq.org/show_bug.cgi?id=4635

------- Additional Comments From [e-mail]  2007-27-04 15:55 -------
Hello,

i might shed some light on this issue :)
While hunting down another bug, i downloaded Framemaker 7.1 from Adobe.
This application is protected by vbox 2.4.x so probably same issue as here.

Well, the error messages popping up at startup of adobe applications result from
an incomplete/corrupted vbox installation.
The vbox installer is called "VSetupT.exe" and contained within main app setup.
It gets executed as post install step.
Due to a failure, only license files get generated and copied (usually into
"c:\program files\common files\vbox\Licenses").
Most of vbox supplementary directories there are empty, vbox client/agent dlls
missing.

The following trace extract nails the problem down:

--- snip post-setup installation step ---
...
002a:Call advapi32.GetNamedSecurityInfoA(0034f994 "c:\Program Files\Common
Files\Vbox\Licenses\FrameMaker
7.1_7.1_CA02.lic",00000001,00000004,00000000,00000000,0034f954,00000000,0034f95c)
ret=004094df
002a:trace:advapi:GetNamedSecurityInfoA c:\Program Files\Common
Files\Vbox\Licenses\FrameMaker 7.1_7.1_CA02.lic 1 4 (nil) (nil) 0x34f954 (nil)
0x34f95c
002a:trace:advapi:GetNamedSecurityInfoW L"c:\Program Files\Common
Files\Vbox\Licenses\FrameMaker 7.1_7.1_CA02.lic" 1 4 (nil) (nil) 0x34f954
(nil) 0x34f95c
002a:Ret  advapi32.GetNamedSecurityInfoA() retval=00000000 ret=004094df
002a:Call
advapi32.AllocateAndInitializeSid(0034f960,00000001,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,0034f950)
ret=00409501
002a:Ret  advapi32.AllocateAndInitializeSid() retval=00000001 ret=00409501
002a:Call advapi32.SetEntriesInAclA(00000001,0034f968,0016fdcc,0034f958)
ret=00409559
002a:fixme:advapi:SetEntriesInAclA 1 0x34f968 0x16fdcc 0x34f958
002a:Ret  advapi32.SetEntriesInAclA() retval=00000078 ret=00409559
002a:Call advapi32.FreeSid(00173488) ret=00409599
002a:Ret  advapi32.FreeSid() retval=00000000 ret=00409599
002a:Call kernel32.RaiseException(e06d7363,00000001,00000003,0034f978) ret=004143b3
002a:trace:seh:raise_exception code=e06d7363 flags=1 addr=0x7b840750
002a:trace:seh:raise_exception  info[0]=19930520
002a:trace:seh:raise_exception  info[1]=0034fd18
002a:trace:seh:raise_exception  info[2]=00434688
002a:trace:seh:raise_exception  eax=7b82bda1 ebx=7b8ab884 ecx=00000000
edx=0034f95c esi=0034f95c edi=0034f8d0
002a:trace:seh:raise_exception  ebp=0034f8b8 esp=0034f854 cs=0073 ds=007b
es=007b fs=0033 gs=003b flags=00200216
002a:trace:seh:call_stack_handlers calling handler at 0x431517 code=e06d7363 flags=1
...
--- snip post-setup installation step ---

Due to advapi32.SetEntriesInAclA() returning ERROR_CALL_NOT_IMPLEMENTED the
installer silently fails install and exits without error.

I tested this on the fly in while running the installer in debugger by patching
SetEntriesInAclA() returning ERROR_SUCCESS in every case.
All following setup steps were executed successfully and the vbox client/agent
dlls get copied too.

These files are necessary for vbox to work (in c:\Program Files\Common
Files\Vbox\Common):

vboxat.dll  vboxm.dll  vboxr.dll  vboxtb.dll  vboxten-us.vboxlm

The application successfully starts up and displays 30-day trial dialog.

Looking at advapi32.SetEntriesInAclW() I see someone already "fixed" it to
return "OK" in all cases - but missed the ANSI version.
To get adobe vbox installations right, apply the following patch:

--- snip dlls/advapi32/security.c ---
DWORD WINAPI SetEntriesInAclA( ULONG count, PEXPLICIT_ACCESSA pEntries,
                               PACL OldAcl, PACL* NewAcl )
{
    FIXME("%d %p %p %p\n",count,pEntries,OldAcl,NewAcl);
     *NewAcl = NULL;
    return ERROR_SUCCESS;
}
--- snip dlls/advapi32/security.c ---

This recipe applies to all adobe vbox protected installations.
Be aware that newer installation/demos from adobe DO NOT use vbox software
protection anymore (SafeCast 3.x now)

Regards