Looking for developers: processing of Windows EMF files for CUPS, NX, Samba and Freedesktop.org printing

10 Jun 2004

      Hi,

my name is Fabian Franz. I am usually hacking for Knoppix. Recently Kurt
showed me NX/NoMachine.com. You can read some more about NX/NoMachine here:

      http://www.linux-kongress.org/2003/abstracts/index.html#3_7_2
      http://www.nomachine.com/

  'X from NoMachine makes remote GUI connections faster than ever before,
   especially over lo-bandwidth/hi-latency links. The remote session can
   be initiated from all major workstation OS-es (Linux, Windows, Mac OS X,
   Solaris) and even PDAs (Zaurus, iPAQ). They can connect truely cross
   platform to Unix-Linux/X11, Windows/RDP and "anything"/VNC. A modem link
   a mere of 40 KBit/sec gives a very comfortable working speed for a KDE
   desktop running Konqueror (file manager), OpenOffice (word prozessor),
   Mozilla (web browser) and KMail (email client). This type of NX link is
   faster to work with than a DSL link with 384 kBit/sec using vanilla X11
   over "ssh -X -C".'

I was fascinated so much by this sofware, that I did some research. I noticed 
that it doesnt yet support printing ...

You are not be interested in the use case I describe next? But you stll find 
the research useful? Then just skip to "==="

The Use Case
-----------------

With NoMachine I have the possiblity to use my machine at home (Windows, 
Linux) from anywhere and from every OS platform (Linux, Windows, Solaris, 
Zaurus, iPAQ, Knoppix) with maximum speed. Even with just 16 kbit/s upstream, 
it has nice performance. 

I login and can work on my machine, with my files and everything is good. :-)

Now I want to print something. I have attached my local printer to my notebook 
and want to print from my remote Linux session.

If my notebook is running linux, everything is fine. The CUPS on the remote 
side is sending the untouched data ("raw printer driver") to the CUPS on my 
side. The local CUPS parses the format, prints with the local driver and my 
printer prints.

So I did not have to configure _anything_ on the remote CUPS daemon. It just 
works out of the box.

Now change the scenario. Assume that now I had to boot my notebook into 
Windows.  The locally attached printer may be proprietary....

So even if I could configure CUPS on the remote side, it still would have no 
driver to print to Windows. 

Windows would pass whatever it gets to the printer and the printer would in 
the best case say "I don't understand this" or print garbage.

But there is one file format every Windows printer driver accepts as input 
(and converts it into the format of his target printer): it is called EMF 
(Enhanced MetaFile). If we could make sure that we send correct EMF to 
Windows, we had a guarantee that it would finally be printed. In fact, one of 
the central jobs of any Windows printer driver is to convert EMF into printer 
specific data.

The task to "reverse engineer" EMF and create a Linux program that outputs
EMF for printing seems to be too complicated at first.

In priciple Windows records all commands that draw something and writes it to 
a (temporary) file. This was done to allow an application to just pass the 
data to Windows and then return back to normal operation. If "direct 
printing" is enabled, the application would have to wait for the printer to 
finally accept the last page. 

This spooling can be used to get hold of EMF files for analyzing. It should   
be noted that EMF is encapsulated then into a *.SPL spool file, which can 
contain several EMF-files and a printjob title.

For more information and a really nice diagram see: 

http://www.printlogger.com/support/articles/msdn/3/index.html

You can also get much information directly from Microsoft or other 
websites. Just use google ;-)

Wouldn't it be nice, if I just printed on my remote session and then Windows 
would use its native driver?

I would print on my remote session with CUPS and CUPS would create EMF (device 
independent extended metafile) and that EMF would be spooled to a SPL
file and that SPL file would be transferred to my local Windows box printer 
and from there it would use the native driver to print my document.

===

The Task
-----------

Spooling EMF to Windows ()instead of raw) to allow usage of the native printer 
driver ...

First it sounds really easy to do. But then you find many problems at second 
sight:

The Problems
-----------------

- There is no program to convert ps to EMF on Linux.
- The MS spool format is proprietary and not easy to understand. (This format 
is called SPL)

But both of these are wrong:

EMF Creation
-----------------

For converting ps to EMF you can use either:

- libEMF.sf.net and pstoedit

or

- OpenOffice.org :-)

While libEMF does only implment a subset of the EMF-fileformat and is 
optimized for graphics, OpenOffice.org's EMF-Output is of perfect quality. 
Kudos to the guys from Star Division, who I believe did this initial job. (If 
I am wrong, please correct me)

So, the first problem is almost solved. Almost, because OpenOffice cannot 
import PS (postscript) or any other vector based fileformat, we have on 
linux. 

But I am sure there are possibilitites, to solve these issues:

Either implement OpenOffice PS-import (We'll need it anyway sooner or later) 
or to extract the EMF-Filter from OpenOffice.org and build it into some 
PS-Application.

Ok, now to the second problem.

SPL File Format
--------------------

- SPL format is proprietairy.

Well, yes. Microsoft gives no information whatsforever about this format, as 
it is due to change in a new version. Well, but when SMB / CIFS was 
implemented I believe it did also change from Release to Release, didn't it?

Fortunatley there are already some "hackers", who have done substantial work 
on the SPL-fileformat, so that its now "known".

Well, but let me start from beginning.

The day before yesterday morning I started my research again, after I got an 
email from Wolfgang Glunz, author of pstoedit, saying that EMF was not like 
the Windows spool-format spl.

So I did some google-groups reseach and I found posts from Feng Yuan. He 
explained that spl was just a small aditional header before and after the EMF 
pages.

Programming resources
-----------------------------

He also pointed to his website http://www.fengyuan.com. Guess what, Feng Yuan 
even did write a book about Windows GDI programming and also had a very nice 
EMF-test-application included. If you want to experiment with EMF, get that 
EMF.zip: It is really worth it. It can even read spl files directly. And it 
prints out the different GDI-commands to print the image, so you can also 
check the "quality" of your EMF-creation tool.

(I'll try to get the book at our local computer science library (If I can 
trust the internet it should be there ;-) ) to look up the chapter about 
printing)

You can "force" your local printer to spool to EMF first and to keep the 
spooled files. (You have to use "EMF NT 1.8" instead of RAW in one of the 
advanced pages and also tick another checkbox to keep the spooled files)

Once I got that spooled spl file, I did look at it with "edit" in binary mode 
and did remove the spl header. And the (one page) spl-file was converted to 
EMF!

What is really nice is, that as I already mentioned, the "hackers" from 
http://undocprint.printassociates.com/spooler/spoolfiles/spl/ did write up, 
what they do know about the spl format and that should be enough to parse or 
create spl files from / to EMF.

"Print Mirror"
----------------

And there is also another very interesting link to some "hacker", who 
published an article on the msdn how to extract EMF files from the spl files.

But not enough, he even started an open source project under the GPL (!) for a 
print mirror utility, that allows one to save the EMF data from the spool 
files.

http://www.angelfire.com/extreme4/vprint/

While it could be a very practical tool to create EMF -> SPL and vice versa 
converters, I also could imagine that having sample GPLed printer code could 
be useful for other developments too. (You'll need the DDK (MS VisualStudio 
DDK (Driver Development Kit)) to compile it, though)

Sources
----------

I have attached some sources I wrote today from the undoc-printing 
specification (You'll need libwine-dev to compile them). 

Thanks go to Christoph Lindemann for publishing that information.

spl_info.c could be easily extended to allow extraction of EMF-files, I just 
was too lazy to do it.

spl_create.c can create valid spl-files from one or more EMF-files.

I tried using an EMF made by OpenOffice.org under linux and spooling that one 
to Windows through normal kprinter / cupsdoprint / smbspool operation.

Guess what? 

Windows did use the native driver installed on the machine. As I did use some 
printer that prints to "FILE", I could see that the output indeed was "HPs 
Adobe Postscript".

So my initial idea indeed does work.

"Experiments" and next steps
-------------------------------------

For first experiment see "sources" above ;-)

Even one step further would be to integrate and experiment with that 
technology in Samba smbspool some more. 

(Jerry Carter from the Samba Team mentioned it could be interesting to change 
to RPC instead of SMB for that).

One could also try to enhance libEMF to allow the whole EMF-"standard". 
(Well, WINE GDI libs and OpenOffice.org source could help)

You can find an overview of EMF at MSDN:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/gdi/metafil... 
- An Overview of the Extended METAFILE Structures.

Or / And to add ps-import to OpenOffice.org.

While pstoedit is nice, the font creation with the current sdl is charcater 
for character, which is not exactly the best for printing ;-) - especially if 
you have such an powerful library as GDI.

Do something cool with "Print Mirror". I for exmaple could imagine that 
testing is much easier that way, as you can directly see the results.

Conclusion
--------------

I believe, when one would combine the results of this "technology analysis", 
it would be possible to:

- Allow spooling of EMF for Samba to accept
  (Dunno if there are szenarios, where this 
   would have advantages)

- Allow a CUPSd to send EMF data to Windows 
  and let Windows use the local printer driver.
  (for the use case mentioned in the beginning)

- With the "PrintMirror" it might be possible to 
  force creation of EMF, as its not always done, yet.

Even if Microsoft decided to change the format of SPL, one could support, I 
believe 3.x/95/98/ME/NT/2000/XP/2003, which should suffice for the next 
years ;-).

Thanks also go to Kurt Pfeifle for reading and helping with this document.

All discussions should CC one mailing list and I would propose samba-technical 
for that.

cu

Fabian

"Bibliography":

http://www.linux-kongress.org/2003/abstracts/index.html#3_7_2 - Explanation of 
NX / NoMachine by Kurt Pfeifle.
http://www.nomachine.com/ - Terminalserver with very efficient compression to 
allow usage over low-bandwidth links
http://www.printlogger.com/support/articles/msdn/3/index.html - Diagrams that 
explain the Windows Spooling Process
http://libEMF.sf.net - Library, which can create EMF together with 
http://www.pstoedit.net
http://www.fengyuan.com  - Book and sample EMF application
http://undocprint.printassociates.com/spooler/spoolfiles/spl/ - The SPL file 
format
http://www.angelfire.com/extreme4/vprint/ - Open Source "Print Mirror" EMF 
extraction utility.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/gdi/metafil... 
- An Overview of the Extended METAFILE Structures.

This posting is dual licensed under the Creative Commons Share Alike 
(http://creativecommons.org/licenses/by-sa/2.0/) and the GNU FDL 
license.

Fabian Franz

Mike McCormack

Wolfgang Glunz

Christoph Lindemann

Kurt Pfeifle

tags

participants (5)