Intercepting GDI calls
Hi, all: I am currently starting a project which tries to run a window application on one (source) machine, and display on another (destination) machine. Of course, the VNC or X11 forwarding technique can achieve the same goal, but I am trying to reduce the bandwidth by "not" transferring the video frame buffer but transfer the GDI function calls instead. The way I can see is try to intercept all the calls inside gdi32.dll, and forward the calls and parameters to remote machine by using some RPC library, and then replay the GDI calls on the destination machine. However, I have two doubts: (1) It seems like the user32.dll is actually handling the window, menu, and cursor. Does the drawing of menu and cursor depends on user-level gdi32.dll? or other kernel-level dlls? (2) It seems that there is no corresponding win32k.sys implementation in wine, and the user-level GDI calls seems to be forwarded to winex11.drv, right? I wander is there any good way or a place inside the source to do this kind of interception? Appreciate any suggestions, Jui-Hao
Jui-Hao Chiang wrote:
Hi, all:
I am currently starting a project which tries to run a window application on one (source) machine, and display on another (destination) machine. Of course, the VNC or X11 forwarding technique can achieve the same goal, but I am trying to reduce the bandwidth by "not" transferring the video frame buffer but transfer the GDI function calls instead. Uhhh... That's kinda how X11 works. I mean, sending drawing and windowing calls as opposed to transferring the entire FB.
What do you hope to accomplish with this? Given what I just told you, why won't X11 suffice?
The way I can see is try to intercept all the calls inside gdi32.dll, and forward the calls and parameters to remote machine by using some RPC library, and then replay the GDI calls on the destination machine.
However, I have two doubts: (1) It seems like the user32.dll is actually handling the window, menu, and cursor.
That's right. Does the drawing of menu and cursor depends on
user-level gdi32.dll? Yes it does. But bear in mind that gdi32 doesn't know how to draw menus and cursors, and other things. It only draws what you tell it to draw. You have to specify what you want to draw in terms of primitive shapes. That's what user32 does. or other kernel-level dlls? What kernel-level DLLs? Wine is entirely user mode. Don't let the presence of "ntoskrnl.exe" fool you; that's done in user mode, too :). (2) It seems that there is no corresponding win32k.sys implementation in wine, and the user-level GDI calls seems to be forwarded to winex11.drv, right? That's correct. (In the future, that might change, especially on a certain OS made by a company named after fruit. But rest assured, there's not going to be a win32k.sys implementation anytime soon.)
I wander is there any good way or a place inside the source to do this kind of interception? If you really want to go through with this, what you want to do is write your own driver, similar to winex11.drv. In fact, I suggest using it as a base. Then you can forward the calls to some other machine (running Windows or some other OS) with RPC or some such mechanism.
Chip
On Fri, Feb 12, 2010 at 8:01 AM, Charles Davis <cdavis(a)mymail.mines.edu> wrote:
Jui-Hao Chiang wrote:
Hi, all:
I am currently starting a project which tries to run a window application on one (source) machine, and display on another (destination) machine. Of course, the VNC or X11 forwarding technique can achieve the same goal, but I am trying to reduce the bandwidth by "not" transferring the video frame buffer but transfer the GDI function calls instead. Uhhh... That's kinda how X11 works. I mean, sending drawing and windowing calls as opposed to transferring the entire FB.
What do you hope to accomplish with this? Given what I just told you, why won't X11 suffice?
The way I can see is try to intercept all the calls inside gdi32.dll, and forward the calls and parameters to remote machine by using some RPC library, and then replay the GDI calls on the destination machine.
However, I have two doubts: (1) It seems like the user32.dll is actually handling the window, menu, and cursor.
That's right. Does the drawing of menu and cursor depends on
user-level gdi32.dll? Yes it does. But bear in mind that gdi32 doesn't know how to draw menus and cursors, and other things. It only draws what you tell it to draw. You have to specify what you want to draw in terms of primitive shapes. That's what user32 does. or other kernel-level dlls? What kernel-level DLLs? Wine is entirely user mode. Don't let the presence of "ntoskrnl.exe" fool you; that's done in user mode, too :). (2) It seems that there is no corresponding win32k.sys implementation in wine, and the user-level GDI calls seems to be forwarded to winex11.drv, right? That's correct. (In the future, that might change, especially on a certain OS made by a company named after fruit. But rest assured, there's not going to be a win32k.sys implementation anytime soon.)
I wander is there any good way or a place inside the source to do this kind of interception? If you really want to go through with this, what you want to do is write your own driver, similar to winex11.drv. In fact, I suggest using it as a base. Then you can forward the calls to some other machine (running Windows or some other OS) with RPC or some such mechanism.
Chip
I also agree that you are basically trying to duplicate X11. It is a lot of work to emulate all gdi32 / user32 calls yourself. Further I doubt you can achieve similar performance as X11. I would investigate in better X11 encryption methods. Have you thought about using NX server? It reduces bandwidth a lot compared to plain X11. Roderick
Charles Davis wrote:
Jui-Hao Chiang wrote:
Hi, all:
I am currently starting a project which tries to run a window application on one (source) machine, and display on another (destination) machine. Of course, the VNC or X11 forwarding technique can achieve the same goal, but I am trying to reduce the bandwidth by "not" transferring the video frame buffer but transfer the GDI function calls instead.
Uhhh... That's kinda how X11 works. I mean, sending drawing and windowing calls as opposed to transferring the entire FB.
What do you hope to accomplish with this? Given what I just told you, why won't X11 suffice?
X11 on a Mac is cumbersome. X11 on Windows is a real pain (I know I work with it on a daily basis) and is downright butt ugly. Opening a new window in Wine or Windows is more desirable. Maybe this can be expanded to support Aqua/Cocoa on MacOSX. I'll agree that this is duplication of the existing X11 code, but the effect is more pleasant to the eye and leads to less user confusion, not to speak of a less expensive solution (I have yet to find a 'free' X11 client that is worth anything on Windows.) One final comment: X11 is the defacto windowing system for Linux/Unix, but let's not forget that the target audience may be any desktop system, Linux/UNIX, MacOSX, Free/OpenBSD, Solaris, and Windows. Thus the built solution should support all of these as close to the Native interface as possible. James McKenzie
James McKenzie skrev:
I'll agree that this is duplication of the existing X11 code, but the effect is more pleasant to the eye and leads to less user confusion, not to speak of a less expensive solution (I have yet to find a 'free' X11 client that is worth anything on Windows.)
You mean X server, not client? (Yes, it's confusing that the machine the user is at is called the server, but that's the way it is...) So, just curious - what is wrong with the Cygwin X server (especially if you use its multi-window mode, which integrates nicely into your Windows desktop)?
On Mon, 15 Feb 2010, Ove Kaaven wrote:
James McKenzie skrev:
I'll agree that this is duplication of the existing X11 code, but the effect is more pleasant to the eye and leads to less user confusion, not to speak of a less expensive solution (I have yet to find a 'free' X11 client that is worth anything on Windows.)
You mean X server, not client? (Yes, it's confusing that the machine the user is at is called the server, but that's the way it is...)
Not really confusing -- it's the piece of the picture providing display services to the program creating output, hence it is the server. Steve Brown sbrown7(a)umbc.edu
Steve Brown skrev:
On Mon, 15 Feb 2010, Ove Kaaven wrote:
James McKenzie skrev:
I'll agree that this is duplication of the existing X11 code, but the effect is more pleasant to the eye and leads to less user confusion, not to speak of a less expensive solution (I have yet to find a 'free' X11 client that is worth anything on Windows.)
You mean X server, not client? (Yes, it's confusing that the machine the user is at is called the server, but that's the way it is...)
Not really confusing -- it's the piece of the picture providing display services to the program creating output, hence it is the server.
Sure it might be confusing, because that's not how the logic goes in the Microsoft world. Over there, the big machine acting as Terminal Server thing is the server, and the Remote Desktop client, which provides the actual display, is the client... while on X11, it's the complete opposite. I'm not going to pretend that it couldn't be confusing.
On Mon, Feb 15, 2010 at 4:05 PM, Ove Kaaven <ovek(a)arcticnet.no> wrote:
Sure it might be confusing, because that's not how the logic goes in the Microsoft world. Over there, the big machine acting as Terminal Server thing is the server, and the Remote Desktop client, which provides the actual display, is the client... while on X11, it's the complete opposite. I'm not going to pretend that it couldn't be confusing.
The whole X11 client/server discussion was touched on in the Unix Haters Handbook chapter on X11. I think that reference proves the merit of the argument. -- Steven Edwards "There is one thing stronger than all the armies in the world, and that is an idea whose time has come." - Victor Hugo
On 18 February 2010 22:08, Steven Edwards <winehacker(a)gmail.com> wrote:
On Mon, Feb 15, 2010 at 4:05 PM, Ove Kaaven <ovek(a)arcticnet.no> wrote:
Sure it might be confusing, because that's not how the logic goes in the Microsoft world. Over there, the big machine acting as Terminal Server thing is the server, and the Remote Desktop client, which provides the actual display, is the client... while on X11, it's the complete opposite. I'm not going to pretend that it couldn't be confusing.
The whole X11 client/server discussion was touched on in the Unix Haters Handbook chapter on X11. I think that reference proves the merit of the argument.
Steven Edwards skrev:
On Mon, Feb 15, 2010 at 4:05 PM, Ove Kaaven <ovek(a)arcticnet.no> wrote:
Sure it might be confusing, because that's not how the logic goes in the Microsoft world. Over there, the big machine acting as Terminal Server thing is the server, and the Remote Desktop client, which provides the actual display, is the client... while on X11, it's the complete opposite. I'm not going to pretend that it couldn't be confusing.
The whole X11 client/server discussion was touched on in the Unix Haters Handbook chapter on X11. I think that reference proves the merit of the argument.
Hmm. Well, I'll take your word for it, I haven't read that and don't plan to, since I'm quite happy with Unix and X11 myself. (Its logic also does make sense to me these days, though that doesn't mean I always understood its architecture. It looks like there's a learning curve to these things, which is why I know it could be confusing to people. Doesn't mean I don't like it. Often, I like confusing stuff.)
On 12 February 2010 06:11, Jui-Hao Chiang <windtracekimo(a)gmail.com> wrote:
I am currently starting a project which tries to run a window application on one (source) machine, and display on another (destination) machine. Of course, the VNC or X11 forwarding technique can achieve the same goal, but I am trying to reduce the bandwidth by "not" transferring the video frame buffer but transfer the GDI function calls instead. The way I can see is try to intercept all the calls inside gdi32.dll, and forward the calls and parameters to remote machine by using some RPC library, and then replay the GDI calls on the destination machine.
Sounds like you're actually trying to reproduce Citrix or Tarantella. (Which do rather better than X11 in many ways.) Not that I have useful advice on doing so ... - d.
On Sun, Feb 14, 2010 at 6:31 PM, David Gerard <dgerard(a)gmail.com> wrote:
On 12 February 2010 06:11, Jui-Hao Chiang <windtracekimo(a)gmail.com> wrote:
I am currently starting a project which tries to run a window application on one (source) machine, and display on another (destination) machine. Of course, the VNC or X11 forwarding technique can achieve the same goal, but I am trying to reduce the bandwidth by "not" transferring the video frame buffer but transfer the GDI function calls instead. The way I can see is try to intercept all the calls inside gdi32.dll, and forward the calls and parameters to remote machine by using some RPC library, and then replay the GDI calls on the destination machine.
Sounds like you're actually trying to reproduce Citrix or Tarantella. (Which do rather better than X11 in many ways.) Not that I have useful advice on doing so ...
In the Wine design this would be very, very inefficient. The main issue is that we lack a DIB engine and due to that we forward those calls to X11 and then read the result back. In case of plain remote rendering this would be very inefficient. In case of Wine you might want something like a local framebuffer which you somehow sync with a remote one... First experiment with NX though but it might suffer from these performance issues. Roderick
participants (8)
-
Charles Davis -
David Gerard -
James McKenzie -
Jui-Hao Chiang -
Ove Kaaven -
Roderick Colenbrander -
Steve Brown -
Steven Edwards