Reducing the header maze (warning: long, unsuitable for children)

[Greg Turner]

I'm working on Ove's comhdr1 & comhdr2 patches.

So far, so good, btw -- I have it compiling against yesterday's CVS 
(recent commits are not folded in yet).  I'll submit after I kick the 
tires a bit and make sure everything seems to be working (otherwise, 
I'll probably post it here and ask for help).

Anyhow, this project has served to remind me just how deep, and how 
confusing, the maze of wine headers can really be.  It's clear to me 
that lots of header inclusions are just historical remnants, that no 
longer serve any purpose, and, conversely, that many implicit header 
dependencies exist that happen to be hidden, and missing (i.e., X.h 
needs Y.h, but all references to X.h happen to occur after Y.h was 
already included, so there's no compile failure).  A third kind of 
unnecessary (but arguably correct, in many cases) dependency is created 
when an include file is pulled in that is already pulled in by some 
prior include...

This negatively affects wine development because (a) it's hard to change 
things, the headers break too easily, and (b) the make process rebuilds 
stuff that doesn't need rebuilding when things change (unavoidable, to 
an extent, of course, but perhaps it's too much right now).

Now, here's my question: couldn't we build a script to automatically fix 
some of these problems?  Imagine this pseudocode as a bash script, 
makefile, or something along those lines:

o gather and create a table of all .c -> .h and .h -> .h dependencies

o for each file:
  o one by one, try removing each reference to a header, and see if the
    affected units still compile.  If they all do, permanently
    remove that reference, but remember it was there

at this point we should have approximately the minimum number of header 
references in the code to compile wine.  But some of the header 
references we eliminated were probably legitimate dependencies,
event though we compiled OK, as mentioned above... so now we:

o for each header, try to compile a "Hello World" which includes just 
that header file.  If it fails (or even generates warnings):
  o for each header reference that was eliminated, try putting it back
    in.  If still failing, try different combinations (this
    could become a really nasty computation with all kinds of
    permutations possible, but if certain simple tricks don't work, we
    can just give up).
  o if we couldn't find any way to fix it, mark the header for human
    review.

This should ensure that each header can actually be run through the 
compiler all by itself -- that is, that it #include's any headers it 
relies on to compile.  Finally...

o A human picks up the pieces and fixes the breakages

o The whole heuristic can start over, now, until the results start to
  look repetitive or trivial...

I can think of some reasons this might not work out so well as I might 
hope, but... what do you guys think?  is such a test meaningful?  Is 
there some tool that already automates this type of investigation, or 
some better way to achieve the same thing?  Would it be, in your 
opinion, too much work for too little benefit?

FWIW, I can probably think of more enjoyable tasks to spend my time 
on... but OTOH I am really tired of waiting for wine to build on my 
aging pc -- anything that could speed up the build process would 
probably save me some from some discomfort, and others too.  

(At least there's no C++ in there; I'm rebuilding my system lately (I 
killed it over the holidays messing with glibc), and building stuff 
like KDE, mozilla, sun-java, and openoffice makes wine look pretty darn 
svelte!  g++ 3.2.1 just crawls; the more "object oriented" the source 
is, the longer it seems to take to compile... :( )

Just "thinking out loud," probably a terrible idea, but let me know what 
you think, anyways...

-- 
gmt

p.s., I was kidding about unsuitable for children... but then, not sure 
why children should want to read this -- or adults, for that matter :P

[Alexandre Julliard]

Greg Turner <gmturner007(a)ameritech.net> writes:
> This should ensure that each header can actually be run through the 
> compiler all by itself -- that is, that it #include's any headers it 
> relies on to compile.

We don't want that, it's not what Windows does. But yes, the whole
obj_*.h stuff is a mess, that's why we should get rid of it all, and
use the exact same include names and dependencies as Windows. Anything
else is a nightmare to maintain.

-- 
Alexandre Julliard
julliard(a)winehq.com

[Greg Turner]

On Tuesday 03 December 2002 04:07 pm, Alexandre Julliard wrote:
> Greg Turner <gmturner007(a)ameritech.net> writes:
> > This should ensure that each header can actually be run through the
> > compiler all by itself -- that is, that it #include's any headers
> > it relies on to compile.
>
> We don't want that, it's not what Windows does. But yes, the whole
> obj_*.h stuff is a mess, that's why we should get rid of it all, and
> use the exact same include names and dependencies as Windows.
> Anything else is a nightmare to maintain.

thanks for all the responses... this makes me think it would be really 
cool to try and visualize the header dependencies in the platform sdk, 
and compare the resulting graph to that of wine.... which is not to say 
that I will bother, unless it falls into my lap... hmm, I wonder if 
ctags or wine's "make depend" could help there...  well, I guess I'll 
spare you all any more ramblings.... but I'll be sure to get back to 
you all if any other random things cross my mind :P

-- 
gmt

[Joerg Mayer]

> Now, here's my question: couldn't we build a script to automatically fix 
> some of these problems?  Imagine this pseudocode as a bash script, 
> makefile, or something along those lines:
>
> o gather and create a table of all .c -> .h and .h -> .h dependencies
>
> o for each file:
>   o one by one, try removing each reference to a header, and see if the
>     affected units still compile.  If they all do, permanently
>     remove that reference, but remember it was there

Last step: Add everything back for systems that
are on a different os (xbsd, x86solaris, reactos)
use older/newer glibcs
whatever I've forgotten

While the idea is good, I think a few people won't be happy with the
result - and correctly so.

 Ciao
     Jörg
--
Joerg Mayer                                          <jmayer(a)loplof.de>
I found out that "pro" means "instead of" (as in proconsul). Now I know
what proactive means.

[Francois Gouget]

On Tue, 3 Dec 2002, Greg Turner wrote:
[...]
> Anyhow, this project has served to remind me just how deep, and how
> confusing, the maze of wine headers can really be.  It's clear to me
> that lots of header inclusions are just historical remnants, that no
> longer serve any purpose, and, conversely, that many implicit header
> dependencies exist that happen to be hidden, and missing (i.e., X.h
> needs Y.h, but all references to X.h happen to occur after Y.h was
> already included, so there's no compile failure).  A third kind of
> unnecessary (but arguably correct, in many cases) dependency is created
> when an include file is pulled in that is already pulled in by some
> prior include...

Many Windows programs expect that they will get the definitions of y.h
when then include x.h because x.h happens to include y.h anyway. So
preserving such includes is important if you do not want to break
Winelib. So as long as your automatic scripts preserve such dependencies
I'm ok with it.

IOW, it's probably not feasible.


-- 
Francois Gouget         fgouget(a)free.fr        http://fgouget.free.fr/
      Any sufficiently advanced bug is indistinguishable from a feature.
                            -- from some indian guy