[Bug 13647] New: DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

Summary: DSM Plugin for UltraVNC 1.0.4 Doesn't Work
           Product: Wine
           Version: 1.0-rc3
          Platform: PC
               URL: http://sourceforge.net/project/showfiles.php?group_id=63
                    887&package_id=60914&release_id=571174
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: -unknown
        AssignedTo: [e-mail]
        ReportedBy: [e-mail]

Created an attachment (id=13629)
 --> (http://bugs.winehq.org/attachment.cgi?id=13629)
relay log

The plugin allows the connection to be encrypted. When selecting the DSM plugin
and trying to connect you receive an error "Connection Failed - Error Reading
Protocol Version". The plugin can be found at ...

http://msrc4plugin.home.comcast.net/~msrc4plugin/

A relay log is attached. A few key lines I noticed from a brief look at the log
for anything obvious...

0009:Ret  rsaenh.CPImportKey() retval=00000000 ret=7eb91d13
0009:Ret  advapi32.CryptImportKey() retval=00000000 ret=10001f3d
0009:Call KERNEL32.MultiByteToWideChar(00000000,00000000,10009ea8 "Error
importing key.",ffffffff,00000000,00000000) ret=7df81852

My guess is this is wine trying to import the rc4.key file the plugin uses. Is
this a crypt related issue?

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

Chad Sikorra  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |download

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

Lei Zhang  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |source

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

--- Comment #1 from Chad Sikorra   2008-09-09 08:10:33 ---
Still happens in Wine-1.1.4.

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

Chad Sikorra  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #13629|0                           |1
        is obsolete|                            |

--- Comment #2 from Chad Sikorra   2009-01-15 08:15:29 ---
Created an attachment (id=18708)
 --> (http://bugs.winehq.org/attachment.cgi?id=18708)
+relay,+seh,+tid

More useful log

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

--- Comment #3 from Chad Sikorra   2009-01-15 08:17:42 ---
Still happens in Wine-1.1.12.

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

unggnu  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #4 from unggnu   2009-01-27 06:49:03 ---
Only the newer MSRC4-Plugins are effected of this bug but the older ones run
only if Wine is changed to "Windows 2000" instead of XP. Using them is no real
solution since they have other bugs.

The source of the plugin is available under
http://msrc4plugin.home.comcast.net/msrc4/MSRC4Plugin122_src.zip.

The error message of the plugin, at least for me, is: "Unknown authentication
scheme!" Of course this message doesn't appears if the viewer is used under
Win.

Could somebody please have a look. Except of this UltraVNC works great in wine
and it is important since SingleClickVNC has no encryption otherwise.

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

Chad Sikorra  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1

--- Comment #5 from Chad Sikorra   2009-01-27 10:12:21 ---
*** This bug has been confirmed by popular vote. ***

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

--- Comment #6 from Chad Sikorra   2009-03-10 08:52:21 ---
I'd be willing to pay someone for their work on getting a fix for this
committed. I'm not sure exactly what the difficulty would be though. It's
pretty much the only thing causing me to boot into my XP VM anymore.

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

--- Comment #7 from Chad Sikorra   2009-03-23 21:16:09 ---
Just a little more info on this. I found out you can create a log for the DSM
plugin. Currently when it fails I get this in the log ...
-----------
03/22/09 18:52:09 : Importing ExponentOfOne KeyBlob
03/22/09 18:52:09 : Importing KEY KeyBlob
03/22/09 18:52:09 : Error importing key. - Last Error = -2146893815 : 
03/22/09 18:52:09 : Reading KeyBlob
03/22/09 18:52:09 : Importing ExponentOfOne KeyBlob
03/22/09 18:52:09 : Importing KEY KeyBlob
03/22/09 18:52:09 : Error importing key. - Last Error = -2146893815 : 
03/22/09 18:52:09 : Key File Read.
03/22/09 18:52:09 : CryptDecrypt failed.
03/22/09 18:52:18 : Shutting Down.
-----------
Error -2146893815 from the log occurs when the plugin calls CryptImportKey
(From crypto.cpp of the plugin's source) ...
-----------
PrintLog((DEST,"Importing KEY KeyBlob"));
//now, we convert the key blob back into a key (internally to the CSP), with
the call to CryptImportKey.
if (! CryptImportKey(hProvider, (const BYTE *)pbBuffer, dwByteCount,
hExchangeKey, 0, hKey)) {
    PrintLog((DEST,"Error importing key."));
    return -1;
}
-----------
MSDN just says that the error code is NTE_BAD_FLAGS which means the dwFlags
parameter was not valid. The point at which it chokes it Wine is in rsaenh.c in
this switch. It hits the default case for the switch and sets NTE_BAD_FLAGS and
the plugin bombs.
-----------
    /*
     * Check if the requested key length is supported by the current CSP.
     * Adjust key length's for DES algorithms.
     */

switch (aiAlgid) {
        case CALG_DES:
            if (dwKeyLen == RSAENH_DES_EFFECTIVE_KEYLEN) {
                dwKeyLen = RSAENH_DES_STORAGE_KEYLEN;
            }
            if (dwKeyLen != RSAENH_DES_STORAGE_KEYLEN) {
                SetLastError(NTE_BAD_FLAGS);
                return (HCRYPTKEY)INVALID_HANDLE_VALUE;
            }
            break;

case CALG_3DES_112:
            if (dwKeyLen == RSAENH_3DES112_EFFECTIVE_KEYLEN) {
                dwKeyLen = RSAENH_3DES112_STORAGE_KEYLEN;
            }
            if (dwKeyLen != RSAENH_3DES112_STORAGE_KEYLEN) {
                SetLastError(NTE_BAD_FLAGS);
                return (HCRYPTKEY)INVALID_HANDLE_VALUE;
            }
            break;

case CALG_3DES:
            if (dwKeyLen == RSAENH_3DES_EFFECTIVE_KEYLEN) {
                dwKeyLen = RSAENH_3DES_STORAGE_KEYLEN;
            }
            if (dwKeyLen != RSAENH_3DES_STORAGE_KEYLEN) {
                SetLastError(NTE_BAD_FLAGS);
                return (HCRYPTKEY)INVALID_HANDLE_VALUE;
            }
            break;

default:
            if (dwKeyLen % 8 || 
                dwKeyLen > peaAlgidInfo->dwMaxLen || 
                dwKeyLen < peaAlgidInfo->dwMinLen) 
            {
                SetLastError(NTE_BAD_FLAGS);
                return (HCRYPTKEY)INVALID_HANDLE_VALUE;
            }
    }
-----------
This is probably the end of the road of my analysis skills for this
issue...haha. Dunno if this helps anyone or not.

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

--- Comment #8 from Chad Sikorra   2009-03-28 00:37:23 ---
Ok...I fixed it...do I get to pay myself now? haha.

I'm not sure if this is the correct fix or not, but I found out that the reason
it failed in the default switch I mentioned above (in rsaenh.c) was because the
dwMaxLen was set to 56 and dwMinLen was set to 40. However, the key length was
being read as 128 so it would fail the check. It was getting the min and max
from RC4 definitions in PROV_ENUMALGS_EX in rsaenh.c (The first occurence of
RC4 - line 163 or so) ....

--------------
{CALG_RC4,       40, 40,   56,0,                    4,"RC4",     24,"RSA Data
Security's RC4"},
--------------

I just changed that to ....

--------------
{CALG_RC4,      128, 40,  128,0,                    4,"RC4",     24,"RSA Data
Security's RC4"},
--------------

Is this just a mistype or copy-paste mistake in that section? It appears that
the first RC2 may have the same type of issue. Can anyone in the know say if
this is the right solution?

Yay, now I can connect to all of the terminals we have without needing to boot
into my XP VM :)

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

--- Comment #9 from Chad Sikorra   2009-03-28 01:21:16 ---
Created an attachment (id=20155)
 --> (http://bugs.winehq.org/attachment.cgi?id=20155)
UltraVNC DSM Fix Patch

Added a patch. First time messing with patches with git, think I got it right.

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

--- Comment #10 from unggnu   2009-03-28 03:49:02 ---
Many thanks, seems to work.
It is not needed anymore to choose "Windows 2000" emulation anymore for old
msrc4 plugin. I am going to check the new one soon.
Encryption is coming :)

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

--- Comment #11 from unggnu   2009-03-28 04:41:59 ---
Hm, but I still got the Unkown Authentication Scheme error with newer MSRC4
plugins.

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

--- Comment #12 from Austin English   2009-03-28 10:48:52 ---
(In reply to comment #8)
> Is this just a mistype or copy-paste mistake in that section? It appears that
> the first RC2 may have the same type of issue. Can anyone in the know say if
> this is the right solution?

Testcases are the best way to find out.

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

--- Comment #13 from Chad Sikorra   2009-03-28 12:56:34 ---
(In reply to comment #11)
> Hm, but I still got the Unkown Authentication Scheme error with newer MSRC4
> plugins.

Yeah, I just setup a newer version of the plugin on a test machine and I get
that too. It's less obvious what the issue is with the newer plugin. When I set
logging for the DSM plugin it doesn't show any errors in the log file and seems
to complete normally. When you look at the vncviewer log file though it shows
...

--------
Started and Winsock (v 2) initialised
bufsize expanded to 4352
Registered connection with app
Connected to <computer-name> port 5900
RFB server supports protocol version 3.6
Connected to RFB server, using protocol version 3.4
Unknown authentication scheme from RFB server: 438174595
--------

I dunno though, I don't see anything obvious in the Wine log.

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

--- Comment #14 from Chad Sikorra   2009-03-28 21:25:16 ---
I'm actually pretty sure the patch I have is only a bandaid on a different
issue. According to this page
(http://msdn.microsoft.com/en-us/library/bb931357(VS.85).aspx), 40/40/56 is
fine for RC2/RC4 key lengths in a few CSP's. So the real issue, I think, would
be to find out why it's not choosing the correct CSP.

Also, concerning the newer plugin, I just noticed that there is a weird issue
with a few characters in the DSM log. On the newer DSM log I get ...

------------
03/28/09 11:46:26 : Importing KEY KeyBlob
03/28/09 11:46:26 : Key File Read.
03/28/09 11:46:26 : Salting D key
03/28/09 11:46:26 : SALTD:¢‡3p^^¾ªHA
03/28/09 11:46:26 :    RFB 003.006

03/28/09 11:46:26 : Salting E key
03/28/09 11:46:26 : SALTE:/®S?›
03/28/09 11:46:31 : Shutting Down.
------------

On that blank line there is also a "^^M". Of course, the ^M is on each line as
well since is a line feed, but why the "^" ? This doesn't appear on a log
generated from an actual Windows box. It may be irrelevant, I just thought it
odd since it appears right after what appears to be the servers RFB protocol
version.

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

--- Comment #15 from Chad Sikorra   2009-07-08 08:23:31 ---
There now appears to be a regression blocking this, or something changed, as it
gets stuck at "Negotiate Protocol Version" or "connecting", when I try to use
the DSM plugin in Wine 1.1.25. I'll run a regression when I get the time,
unless someone else has some time to kill :)

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

[e-mail]  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #16 from [e-mail]   2009-09-06 16:48:48 ---
(In reply to comment #15)
> There now appears to be a regression blocking this, or something changed, as it
> gets stuck at "Negotiate Protocol Version" or "connecting", when I try to use
> the DSM plugin in Wine 1.1.25. I'll run a regression when I get the time,
> unless someone else has some time to kill :)

have tested with wine 1.1.29 - can confirm that running ultravnc 1.0.6 with
"windows xp" as a global wine setting and connecting using msrc encryption does
not work.  However, running a "windows 2000" setting for vncviewer.exe
(winecfg) does work.  Great.  FTP does crash however after transfer for one
file though.

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

--- Comment #17 from unggnu   2009-11-08 14:36:19 ---
This issue seems to be fixed with current Wine 1.1.32. Encryption seems to work
without switching to Windows 2000.

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #18 from Austin English   2009-11-08 15:44:47 ---
(In reply to comment #17)
> This issue seems to be fixed with current Wine 1.1.32. Encryption seems to work
> without switching to Windows 2000.

Fixed then.

[Bug 13647] DSM Plugin for UltraVNC 1.0.4 Doesn't Work

http://bugs.winehq.org/show_bug.cgi?id=13647

Alexandre Julliard  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #19 from Alexandre Julliard   2009-11-13 12:42:58 ---
Closing bugs fixed in 1.1.33.