[Bug 11539] Anycount 6.0 refuses to start

http://bugs.winehq.org/show_bug.cgi?id=11539

--- Comment #2 from Anastasius Focht   2008-02-13 14:27:38 ---
Created an attachment (id=10755)
 --> (http://bugs.winehq.org/attachment.cgi?id=10755)
patch which fixes VMWare backdoor check (false positive)

Hello,

although the error dialog caption states "AIT Protection System", the target is
wrapped with infamous Themida/WinLicense 1.9.x Software Protection.
They probably added some custom dialog resources and rechristened it as their
own protection system ;-)

Anyway ... the error message itself is correct.
The packaging developer either disabled Themida's builtin "Virtual Machine
Compatibility Mode" on purpose to artificially limit the software to real
windows machines or had no clue.

Debugging was a bit nasty (lots of anti-debug tricks and obfuscated/vm code)
but it basically boils down to following checks ... I commented the snippets
for your pleasure.

1.) checks for Virtual PC

--- snip ---
... <obfuscated code>
db 0F, 07, 0B, 64 ; causes illegal instruction exception if not running under
VM
                  ; if running under Virtual PC, internally handled and
                  ; exception never seen by app
POP DWORD PTR FS:[0]
ADD ESP,4
MOV DWORD PTR SS:[EBP+71B0225],EBX  ; store result for later examination
... <obfuscated code>
--- snip ---

SEH for snippet:

--- snip ---
MOV ECX,DWORD PTR SS:[ESP+0C] ; ContextRecord
MOV DWORD PTR DS:[ECX+0A4],-1 ; ctx->Ebx, set not running in Virtual PC
ADD DWORD PTR DS:[ECX+0B8],4 ; ctx->Eip += 4 (skip Virtual PC opcode sequence)
XOR EAX,EAX ; EXCEPTION_CONTINUE_EXECUTION
RETN
--- snip ---

Ok, this one is gracefully passed by wine.

2.) checks for VMWare:

--- snip ---
... <obfuscated code> ; eax = 0x564D5868 ('VMXh' VMWare magic number)
                      ; dx = 0x5658 ('VX' VMWare backdoor port)
                      ; ecx = 0xA (VMWare backdoor 'get version')
IN EAX,DX  ; opcode: 0xED -> cause privileged instruction exception in ring3
CMP EBX,564D5868 ; backdoor magic value, changed if running under VMWare
JNE SHORT no_vmware
MOV DWORD PTR SS:[EBP+71B1455],1 ; VMWare detected
no_vmware:
POP DWORD PTR FS:[0]
ADD ESP,4
... <obfuscated code>
--- snip ---

SEH for snippet (obfuscated):

--- snip ---
MOV EBX,DWORD PTR SS:[ESP+0C] ; ContextRecord
PUSH EBP
CALL $+5
POP EBP
SUB EBP,71E3EA3
MOV EAX,DWORD PTR DS:[EBX+0B8] 
LEA EAX,[EBP+71E3F4B]
MOV DWORD PTR DS:[EBX+0B8],EAX ; ctx->Eip ('no_vmware' address of previous
snippet)
POP EBP
XOR EAX,EAX ; EXCEPTION_CONTINUE_EXECUTION
RETN
--- snip ---

Unfortunately wine gets it wrong with __wine_emulate_instruction(), "eating"
this exception internally.
The app SEH never executes which leads to "Virtual Machine detected" result.
Attached patch fixes this.
It looks for magic VMWare numbers if the special "IN EAX,DX" is to be emulated.
In that case the instruction is not emulated and the exception is passed down
to next handler in SEH chain.

3.) General checks by using IDTR, GDTR and LDTR register values

The instructions to retrieve these values (selectors) are non-privileged in
ring3 (not causing exception).

Example:

--- LDT ---
... <obfuscated code>
sldt word ptr ss:[esp] ; store segment selector from the LDTR in a 16bit memory
location
... <obfuscated code>
pop eax
... <obfuscated code>
or ax, ax
jz no_vm
--- LDT ---

On real windows machine the LDT selector value is 0x0000.
In Virtual Machines the value is usually something different - unfortunately
the same applies to linux ;-(

Due to the nature of these instructions (non-privileged) this problem can't be
fixed in wine.
Ask the Anycount software vendor (Advanced International Translations) to
enable Themida's Virtual Machine compatibility mode or just don't buy/support
their products.

Anyway - at least the special check for VMWare backdoor can be fixed in wine
and should be handled properly.

Regards