HtmlHelp status, winecfg and SoC proposal

[Jacek Caban]

Hello.

As you may have noticed, I've been working on HtmlHelp lately. Currently 
it should generally work, but still there is a lot to do. HTML should be 
correctly displayed and content tab should work. I was thinking about 
making more use of it in Wine. I think winecfg could benefit from it. We 
could easily add context help to winecfg using HtmlHelp API. Even just 
integrating a part of Wine User's Guide would be a good start. The main 
problem is building chm file (that is the file format of HtmlHelp). 
Microsoft has a HtmlHelp compiler (hhc) for it. We'd need something 
similar  in Wine. AFAIK there is no good open source replacement. 
HtmlHelp maker (hhc, see [1]) doesn't create internal files (parts of 
chm that describes stuff like index or default topic) and it's GPLed so 
it's useless for Wine (unless author would relicense it for us). I think 
it would be a good project for Google Summer of Code. The task would be 
to write a hhc replacement and add a help option to winecfg. hhc 
replacement (say whhc) would have to be a plain UNIX tool (it means a 
bit of code duplication with itss.dll, just like we do in widl) so we 
could use it during compilation. I think its difficulty is good for SoC. 
Compressing code may be integrated from some other project. The 
remaining parts are code of parser of files describing chm and a little 
winecfg hacking. What do you think?

Jacek

[1] http://savannah.nongnu.org/projects/hhm

[Saulius Krasuckas]

* On Sun, 4 Mar 2007, Jacek Caban wrote:
> HtmlHelp maker (hhc, see [1]) doesn't create internal files (parts of 
> chm that describes stuff like index or default topic) and it's GPLed so 
> it's useless for Wine (unless author would relicense it for us).

Quoting Paul Wise resume from the [1]:

| Started off hacking on frhed (a windows hex editor)
| Made some lame/unfinished patches for POV-Ray
| Wrote some tiny windows software
| Wrote chmdeco/hhm/chmspec
| Been hacking on activismo for a while
| Looking forward to getting ADSL & contributing to xCHM, possibly WINE, GNOME, random projects.
| I'd also like to become a debian developer at some stage. 

Paul seems to be excited about Wine, so hhm relicensing should sound OK 
for him, IMHO.  But only if proved it isn't breaking M$ licences [2], 
right?
> I think it would be a good project for Google Summer of Code. The task 
> would be to write a hhc replacement and add a help option to winecfg. 
> hhc replacement (say whhc) would have to be a plain UNIX tool (it means 
> a bit of code duplication with itss.dll, just like we do in widl) so we 
> could use it during compilation. I think its difficulty is good for SoC. 
> Compressing code may be integrated from some other project. The 
> remaining parts are code of parser of files describing chm and a little 
> winecfg hacking. What do you think?

Nice idea.  Maybe even Paul himself would join that, as he states in his 
blog [3] he needs "Employment, bux, moolah, work".
(I've Bcc-ed Paul in his message)
> [1] http://savannah.nongnu.org/projects/hhm
>   [2] http://msdn.microsoft.com/library/en-us/htmlhelp/html/hwhtmlhelpend-userlice...
>   [3] http://bonedaddy.net/pabs3/

[Paul Wise]

[please CC me on replies, I'm not subscribed to wine-devel]

On Mon, 2007-03-05 at 19:30 +0200, Saulius Krasuckas wrote:
> * On Sun, 4 Mar 2007, Jacek Caban wrote:
> > HtmlHelp maker (hhc, see [1]) doesn't create internal files (parts of 
> > chm that describes stuff like index or default topic) and it's GPLed so 
> > it's useless for Wine (unless author would relicense it for us).
>
> Quoting Paul Wise resume from the [1]:

Eeep, haven't updated that in years. Just deleted it and replaced with a
link to my homepage.
> Paul seems to be excited about Wine, so hhm relicensing should sound OK 
> for him, IMHO.

I've previously offered to relicence the hhm code for inclusion in both
chmlib and libmspack, but it seems neither project took up this offer. I
don't think the hhm code belongs anywhere other than in one of them. The
current version was more of a proof-of-concept, I wanted to push the
code into a chm reading/writing library. IIRC, wine's itss.dll uses
libmspack, so perhaps it should go there?

Also, if it does get included, I'd *STRONGLY* request/suggest that the
code be modified to also include a README.hhm.txt (or similar) file
saying something like "This file was not produced by Microsoft's
HtmlHelp compiler, it may have incompatibilities preventing it from
being used on Windows." I think this is an important inclusion in any
non-MS compiler, one which I haven't got around to adding to hhm yet. 

I've also previously (2003) pointed wine-devel at chmspec and offered to
relicence chmdeco code for use in a wine version of hh.exe. No-one was
interested though, saying that a CHM viewer was already implemented and
then deteriorating into a thread about which browser to embed and
forking KHTML and Mozilla and so on.
> But only if proved it isn't breaking M$ licences [2], right?

The licence says this:
> * Limitations on Reverse-Engineering, Decompilation, and Disassembly.
> You may not reverse- engineer, decompile, or disassemble the SOFTWARE
> PRODUCT, except and only to the extent that such activity is expressly
> permitted by applicable law notwithstanding this limitation.

As written in the chmspec colophon, I didn't do any disassembly of
Microsoft stuff, only black-box format discovery. Mainly this was by
sending different inputs to Microsoft's hhc and observing the output, as
well as looking at existing samples of CHM files. I imagine that is
probably "reverse-engineering" according to Microsoft (although they
haven't contacted me at all, so maybe not).

http://chmspec.nongnu.org/latest/Colophon.html
> Nice idea.  Maybe even Paul himself would join that, as he states in his 
> blog [3] he needs "Employment, bux, moolah, work".
> (I've Bcc-ed Paul in his message)

I'm not a student, so I'm not eligible to implement that project as part
of the SoC. If people want to pay me privately, that would be great
since I'm in need of work at the moment. I'm happy to answer questions
that come up in the SoC in relation to chmspec though.

Frankly I haven't done anywhere near enough reverse engineering of the
internal CHM formats to even be close to thinking about implementing a
full-blown CHM compiler that would create correct CHMs. There is enough
info for CHM readers like chmdeco, xCHM and so on to be written, but not
for compilers. Patches to chmspec are welcome though :)

Why does wine want a CHM compiler anyway? Can't Microsoft's one be used?
Microsoft's one isn't distributed with the OS anyway.

Also, Matthew's ITSF (the container format built and read by itss.dll)
spec has a few bugs that I did a bit of work on while I was sailing,
haven't done anything recently on them though, I'm hoping to get to that
soon.

Also, there is some bug in the chmlib/hhm/lzx_comp combination that
causes xCHM and other chmlib based CHM viewers to crash when accessing
CHMs created using hhm (linked against lzx_comp), this needs
investigating and hhm/lzx_comp should not be used until it is.

-- 
bye,
pabs

http://pabs.zip.to

[Joris Huizer]

--- Paul Wise <pabs3(a)bonedaddy.net> wrote:
> The licence says this:
> > * Limitations on Reverse-Engineering,
> > Decompilation, and Disassembly.
> > You may not reverse- engineer, decompile, or
> > disassemble the SOFTWARE
> > PRODUCT, except and only to the extent that such
> > activity is expressly
> > permitted by applicable law notwithstanding this
> > limitation.
>
> As written in the chmspec colophon, I didn't do any
> disassembly of
> Microsoft stuff, only black-box format discovery.
> Mainly this was by
> sending different inputs to Microsoft's hhc and
> observing the output, as
> well as looking at existing samples of CHM files. I
> imagine that is
> probably "reverse-engineering" according to
> Microsoft (although they
> haven't contacted me at all, so maybe not).
>
> http://chmspec.nongnu.org/latest/Colophon.html

That sounds like the same way wine and samba are
developed, which is called "clean-room
reverse-engineering", and which should be legal. If it
were not, using this code in wine wouldn't make
anything worse ;-)

regards,
Joris


 
____________________________________________________________________________________
Be a PS3 game guru.
Get your game face on with the latest PS3 news and previews at Yahoo! Games.
http://videogames.yahoo.com/platform?platform=120121

[Jacek Caban]

Hi Paul,

Paul Wise wrote:
> > Paul seems to be excited about Wine, so hhm relicensing should sound OK 
> > for him, IMHO.
>
> I've previously offered to relicence the hhm code for inclusion in both
> chmlib and libmspack, but it seems neither project took up this offer. I
> don't think the hhm code belongs anywhere other than in one of them. The
> current version was more of a proof-of-concept, I wanted to push the
> code into a chm reading/writing library. IIRC, wine's itss.dll uses
> libmspack, so perhaps it should go there?

Yes, it should go to itss.dll to support writing mode in storages. But
we can't use itss in hhc replacement as it has to be useful for Wine
compilation. It means that we need plain UNIX app. That's the code
duplication I wrote about.
> Also, if it does get included, I'd *STRONGLY* request/suggest that the
> code be modified to also include a README.hhm.txt (or similar) file
> saying something like "This file was not produced by Microsoft's
> HtmlHelp compiler, it may have incompatibilities preventing it from
> being used on Windows." I think this is an important inclusion in any
> non-MS compiler, one which I haven't got around to adding to hhm yet.

Then we could add such README about whole Wine :-) Our aim is to improve
things to be useful, not to write something and tell users to not use it
because it sucks...
> I've also previously (2003) pointed wine-devel at chmspec and offered to
> relicence chmdeco code for use in a wine version of hh.exe. No-one was
> interested though, saying that a CHM viewer was already implemented and
> then deteriorating into a thread about which browser to embed and
> forking KHTML and Mozilla and so on.

Things have changed. Now Wine has MSHTM/WebBrowser implementation that
make it possible to support HtmlHelp.
> > But only if proved it isn't breaking M$ licences [2], right?
>
> The licence says this:
> > * Limitations on Reverse-Engineering, Decompilation, and Disassembly.
> > You may not reverse- engineer, decompile, or disassemble the SOFTWARE
> > PRODUCT, except and only to the extent that such activity is expressly
> > permitted by applicable law notwithstanding this limitation.
>
> As written in the chmspec colophon, I didn't do any disassembly of
> Microsoft stuff, only black-box format discovery. Mainly this was by
> sending different inputs to Microsoft's hhc and observing the output, as
> well as looking at existing samples of CHM files. I imagine that is
> probably "reverse-engineering" according to Microsoft (although they
> haven't contacted me at all, so maybe not).
>
> http://chmspec.nongnu.org/latest/Colophon.html

That's fine for sure. Thanks for your work, it was very helpful to me!
> Frankly I haven't done anywhere near enough reverse engineering of the
> internal CHM formats to even be close to thinking about implementing a
> full-blown CHM compiler that would create correct CHMs. There is enough
> info for CHM readers like chmdeco, xCHM and so on to be written, but not
> for compilers. Patches to chmspec are welcome though :)

I have to disagree. Surely it's enough to implement a compiler that will
work with Wine's HtmlHelp. Also it should be possible to implement a
compiler that will work with native hhctrl.ocx. Even if not everything
is documented, dummy values should be good enough for most uncovered
things. Also more things that will be needed may be discovered during SoC.
> Why does wine want a CHM compiler anyway? Can't Microsoft's one be used?
> Microsoft's one isn't distributed with the OS anyway.

If we want to include chm files to Wine (like help for winecfg) we have
to be able to compile them. Perhaps some apps ported with winelib would
benefit from chm compiler.

Thanks,
    Jacek

[Paul Wise]

On Tue, 2007-03-13 at 20:13 +0100, Jacek Caban wrote:
> Yes, it should go to itss.dll to support writing mode in storages. But
> we can't use itss in hhc replacement as it has to be useful for Wine
> compilation. It means that we need plain UNIX app. That's the code
> duplication I wrote about.

AFAIK, itss.dll doesn't know anything about internal CHM formats, it
just deals with the InfoTech Storage format (container format for CHMs),
so no need for code duplication at all.

For a wine version of hha.dll/hhctrl.ocx and so on, there is always the
possibility of creating a libhtmlhelp or something like that that can
read and write the internal CHM formats, that uses a write-enabled
libmspack or chmlib. Then use that library in wine's version of
hha.dll/hhctrl.ocx/etc as well as the compiler, no need for code
duplication at all. The current chmdeco code could form the initial
decoding part of that library and write support could form around that.
> > Also, if it does get included, I'd *STRONGLY* request/suggest that the
> > code be modified to also include a README.hhm.txt (or similar) file
> > saying something like "This file was not produced by Microsoft's
> > HtmlHelp compiler, it may have incompatibilities preventing it from
> > being used on Windows." I think this is an important inclusion in any
> > non-MS compiler, one which I haven't got around to adding to hhm yet.
>
> Then we could add such README about whole Wine :-) Our aim is to improve
> things to be useful, not to write something and tell users to not use it
> because it sucks...

To be clear, this file would not be visible to users unless they unpack
the files with extract_chmLib or istorage.exe or something. The idea was
to document the fact that those files are non-Microsoft, so if there are
issues with those files later, the person investigating those issues
will immediately know that the file is non-Microsoft and then be able to
direct any bug reports to the appropriate place. So it wouldn't be
telling users not to use it, it would be telling recipients of
hhm-created files where to go in case of problems.
> Things have changed. Now Wine has MSHTM/WebBrowser implementation that
> make it possible to support HtmlHelp.

That is good to hear.
> I have to disagree. Surely it's enough to implement a compiler that will
> work with Wine's HtmlHelp. Also it should be possible to implement a
> compiler that will work with native hhctrl.ocx. Even if not everything
> is documented, dummy values should be good enough for most uncovered
> things. Also more things that will be needed may be discovered during SoC.

I suppose it may be possible, I'd still suggest adding a README.hhm.txt
to CHM files produced by hhm to warn that it may still be using dummy
values. If you want to implement that I'm happy to provide explanations
of any unclear bits in chmspec and accept chmspec bugs or provide other
help on the project.
> If we want to include chm files to Wine (like help for winecfg) we have
> to be able to compile them. Perhaps some apps ported with winelib would
> benefit from chm compiler.

Ok, that makes sense. I don't really understand the desire to use CHM
files in Wine, that isn't relevant to this conversation though.

-- 
bye,
pabs

http://pabs.zip.to