Hi, Alexandre did not like the needed changes required to add support for this but I figured I would throw it out for debate anyway in case anyone else can make him change his mind (duck). The issue is he does not like needing a single header per-program or per dll. After doing some research it seems to me that even MSVC does not support multiple PCH files per project so I don't know how it could be fixed in gcc to do it Microsoft has not figured it out.
problem: ccache works great for speeding up compiles but seems to still have issue with some header changes. Also the ReactOS guys really want to add pch support to the Wine dlls that are shared so the job falls to me to sell it to Wine.
Solution: Below are the benchmarks from my box and a patch. Simply patch your tree and then ./configure and cd to the regedit folder and do a 'time make','make clean','make pch', and 'time make' if you want to see the speedup on your box. I expect we could get a 2x speedup in fresh compiles of Wine with pch support
Regedit compile without pch
real 0m5.632s user 0m5.000s sys 0m0.524s
Regedit compile with pch
real 0m2.797s user 0m2.352s sys 0m0.428s
-- Steven Edwards - ReactOS and Wine developer
"There is one thing stronger than all the armies in the world, and that is an idea whose time has come." - Victor Hugo
Steven Edwards wrote:
Alexandre did not like the needed changes required to add support for this but I figured I would throw it out for debate anyway in case anyone else can make him change his mind (duck). The issue is he does not like needing a single header per-program or per dll. After doing some research it seems to me that even MSVC does not support multiple PCH files per project so I don't know how it could be fixed in gcc to do it Microsoft has not figured it out.
Well, having a single header file to include indeed looks not nice. This would grow and grow as more modules are added and eventually you would be adding the entire SDK headers to every source file indirectly. It is also not the way MSVC does do it. Instead the preparser or something seems to create a separate pch file from all the specifically included headers in that project and it seems smart enough to allow for different headers in different source files that can conflict if used together. I'm not sure how you would suppose to avoid possible conflicts of headers in this way. Maybe you consider maintaining one single precompile.h file per module but that's one more file to maintain and it still won't solve the issue of Conflicting header files used in different source files inside that module.
The Wine/MS SDK/crt headers are already a nightmare to combine sometimes and your single header would only increase that issue. And Wine attempting to be compilable on many different systems besides standard Linux, has certainly a lot more possible difficulties with header conflicts than most other systems.
problem: ccache works great for speeding up compiles but seems to still have issue with some header changes. Also the ReactOS guys really want to add pch support to the Wine dlls that are shared so the job falls to me to sell it to Wine.
I've had issues with header changes with MSVC precompiled headers too. Sometimes the only solution is to disable precompiled headers and delete the pch file. For fairness I have to say that I'm still using MSVC 6 so there is a good chance that more recent MSVC systems have better dependency tracking in precompiled headers.
Rolf Kalbermatter
On Thu, 12 Jan 2006, Rolf Kalbermatter wrote:
I've had issues with header changes with MSVC precompiled headers too. Sometimes the only solution is to disable precompiled headers and delete the pch file. For fairness I have to say that I'm still using MSVC 6 so there is a good chance that more recent MSVC systems have better dependency tracking in precompiled headers.
Disabling precompiled headers is the first thing I do when starting a new project. We are using Visual C++ 2003/7.1, which often fails if you are building from a network share (this is more or less unsupported though, see http://groups.google.com/groups?hl=en&lr=lang_en&ie=UTF-8&oe=UTF...).
Peter Åstrand wrote:
Disabling precompiled headers is the first thing I do when starting a new project. We are using Visual C++ 2003/7.1, which often fails if you are building from a network share (this is more or less unsupported though, see http://groups.google.com/groups?hl=en&lr=lang_en&ie=UTF-8&oe=UTF...).
We've been using PCH with GCC for a long time in ReactOS, it's been working very well and reliable. Compiling ReactOS is *a lot* faster with PCH enabled.
I don't really understand what the problem is. Nothing really should change except that all includes are grouped into one header file. It works the same with MSVC. The only difference is that dependency tracking needs to use one minor hack so it works properly with MSVC. Even ancient versions of GCC don't have a problem with it and should compile everything properly without any additional hacks (without PCH support of course).
- Thomas
Thomas Weidenmueller wrote:
We've been using PCH with GCC for a long time in ReactOS, it's been working very well and reliable. Compiling ReactOS is *a lot* faster with PCH enabled.
Basically what I get in MSVC 6 when modifing a header somewhere and PCH is enabled is that MSVC keeps mocking about errors in the headers, my change was supposed to fix. Disabling PCH (and sometimes deleting the *.pch file) always fixes those issues.
The issues about getting all the necessary headers in one single include file is, that sometimes certain headers conflict but one is used in one particular source file and another one in another source file. Of course you can always fix those issues by adding precompiler conditions around certain declarations or even complete include declarations, and that would be the prefered solution but sometimes this can't be easily done because you might break compilation on a different platform or with a different header set/compiler.
I think PCH is fine as long as you do not need to modify anything about the source files and the preprocessor creating the pch file is smart enough to correctly track all possible dependancies.
Apparently this seems not possible: MSVC seems to have trouble tracking dependancies reliable but does not require to put all used headers in a special include file, and gcc does require this modification but does not have trouble to track proper dependancies (or I have never understood how PCH is supposed to work under MSVC, but for me not having projects containing million lines of c code, disabling PCH always was the most simple and reliable solution).
Rolf Kalbermatter
On Thu, Jan 12, 2006 at 09:02:42PM +0100, Rolf Kalbermatter wrote:
Thomas Weidenmueller wrote:
We've been using PCH with GCC for a long time in ReactOS, it's been working very well and reliable. Compiling ReactOS is *a lot* faster with PCH enabled.
Basically what I get in MSVC 6 when modifing a header somewhere and PCH is enabled is that MSVC keeps mocking about errors in the headers, my change was supposed to fix. Disabling PCH (and sometimes deleting the *.pch file) always fixes those issues.
Unless, of course, your filesystem is samba-mounted from a linux box, when PCH tends to cause internal compiler errors.
David
On Thu, Jan 12, 2006 at 09:02:42PM +0100, Rolf Kalbermatter wrote:
Thomas Weidenmueller wrote:
We've been using PCH with GCC for a long time in ReactOS,
it's been
working very well and reliable. Compiling ReactOS is *a lot* faster with PCH enabled.
Basically what I get in MSVC 6 when modifing a header somewhere and PCH is enabled is that MSVC keeps mocking about errors in the headers, my change was supposed to fix. Disabling PCH (and sometimes deleting the *.pch file) always fixes those issues.
Unless, of course, your filesystem is samba-mounted from a linux box, when PCH tends to cause internal compiler errors.
Well, I finally read a bit about how precompiled headers are supposed to work and it seems MSVC does work in a similar way than gcc. Basically you can only have one pch file per working unit.
My problem was that when you create a new project in MSVC 6, PCH is by default enabled and set to automatic but no specific header file is filled in in the project settings. What this seems to do is taking some (which?) header files and putting them in an automatically generated pch file in your project output directory. Of course for more than 1 source file in a project this is likely each time different so build times will be even worse than without PCH. Also it seems that header dependancy tracking isn't reliable in such a setup.
From some explanations, I gathered that the only useful PCH setting in MSVC is as well to use one specific pch include file and even
more specifically configure one source file to create that pch file and all others only to use it, otherwise MSVC tends to recompile the pch file on some arbitrary conditions multiple times for different source files, basically reverting the purpose of PCH more or less or even make it worse.
One thing to watch out also is that any statements in a source file before the specifically set precompiled header file are completely, and without any indication (well except the strange resulting compile errors of course) ignored.
Rolf Kalbermatter
On Fri, Jan 13, 2006 at 10:47:43AM +0100, Rolf Kalbermatter wrote:
On Thu, Jan 12, 2006 at 09:02:42PM +0100, Rolf Kalbermatter wrote:
Thomas Weidenmueller wrote:
We've been using PCH with GCC for a long time in ReactOS,
it's been
working very well and reliable. Compiling ReactOS is *a lot* faster with PCH enabled.
Basically what I get in MSVC 6 when modifing a header somewhere and PCH is enabled is that MSVC keeps mocking about errors in the headers, my change was supposed to fix. Disabling PCH (and sometimes deleting the *.pch file) always fixes those issues.
Unless, of course, your filesystem is samba-mounted from a linux box, when PCH tends to cause internal compiler errors.
Well, I finally read a bit about how precompiled headers are supposed to work and it seems MSVC does work in a similar way than gcc. Basically you can only have one pch file per working unit.
My problem was that when you create a new project in MSVC 6, PCH is by default enabled and set to automatic but no specific header file is filled in in the project settings. What this seems to do is taking some (which?) header files and putting them in an automatically generated pch file in your project output directory. Of course for more than 1 source file in a project this is likely each time different so build times will be even worse than without PCH. Also it seems that header dependancy tracking isn't reliable in such a setup.
From some explanations, I gathered that the only useful PCH setting in MSVC is as well to use one specific pch include file and even
more specifically configure one source file to create that pch file and all others only to use it, otherwise MSVC tends to recompile the pch file on some arbitrary conditions multiple times for different source files, basically reverting the purpose of PCH more or less or even make it worse.
One thing to watch out also is that any statements in a source file before the specifically set precompiled header file are completely, and without any indication (well except the strange resulting compile errors of course) ignored.
Hmm, perhaps use a CFLAGS "-include autogenerated.h" trick.
Generate "autogenerated.h" by collecting all possible includes via a script on "make depend", then force it into every file with -include autogenerated.h (previously compiled to pch).
Has the advantage that the magic is all in the buildsystem, no source files need to be touched.
Ciao, Marcus
Hmm, perhaps use a CFLAGS "-include autogenerated.h" trick.
Generate "autogenerated.h" by collecting all possible includes via a script on "make depend", then force it into every file with -include autogenerated.h (previously compiled to pch).
Has the advantage that the magic is all in the buildsystem, no source files need to be touched.
This solution has been starting to form in my mind as well. It is even portable to MSVC it seems with a few small modifications to winapi/msvcmaker.
For platforms not supporting PCH the build system would simply behave as is. Watch out though that you probably do not want to include every possible public header file, due to some conflicts. But for the majority of windows.h and friends this could be a good solution.
Rolf Kalbermatter
Hi,
On 1/13/06, Rolf Kalbermatter rolf.kalbermatter@citeng.com wrote:
This solution has been starting to form in my mind as well. It is even portable to MSVC it seems with a few small modifications to winapi/msvcmaker.
I came up with a much more generic solution that will work on all compilers and does not cause a dependancy issue. It requires some minor code adjustments and there is a slight issue if you have errors in linking. The error location is wrong so you need to have a way to disable this if you have a real error to track the problem down, but once I fixed ReactOS shell32.dll to use my hack my compile times on gcc4 went from 30+ secs down to 9sec. MSVC went from 22 to less than 1 secound.
In short each module would have its own autogenerated master C file that would include all of the others like the one I have attached. Minus the #define hacks. I personally think its more of a nasty hack but a configure switch could be added like
--enable-compilation-units
Because each header is only processed once, it give the same effect as PCH.
-- Steven Edwards - ReactOS and Wine developer
"There is one thing stronger than all the armies in the world, and that is an idea whose time has come." - Victor Hugo
Just FYI here are some tests I made in November:
http://www.reactos.org/archives/public/ros-dev/2005-November/006273.html
Casper
-----Original Message----- From: wine-devel-bounces@winehq.org [mailto:wine-devel-bounces@winehq.org] On Behalf Of Steven Edwards Sent: 13. januar 2006 17:01 To: Rolf Kalbermatter Cc: wine-devel@winehq.org; Marcus Meissner Subject: Re: pch support
Hi,
On 1/13/06, Rolf Kalbermatter rolf.kalbermatter@citeng.com wrote:
This solution has been starting to form in my mind as well. It is even portable to MSVC it seems with a few small modifications to
winapi/msvcmaker.
I came up with a much more generic solution that will work on all compilers and does not cause a dependancy issue. It requires some minor code adjustments and there is a slight issue if you have errors in linking. The error location is wrong so you need to have a way to disable this if you have a real error to track the problem down, but once I fixed ReactOS shell32.dll to use my hack my compile times on gcc4 went from 30+ secs down to 9sec. MSVC went from 22 to less than 1 secound.
In short each module would have its own autogenerated master C file that would include all of the others like the one I have attached. Minus the #define hacks. I personally think its more of a nasty hack but a configure switch could be added like
--enable-compilation-units
Because each header is only processed once, it give the same effect as PCH.
-- Steven Edwards - ReactOS and Wine developer
"There is one thing stronger than all the armies in the world, and that is an idea whose time has come." - Victor Hugo
Just FYI here are some tests I made in November:
http://www.reactos.org/archives/public/ros-dev/2005-November/006273.html
Casper
-----Original Message----- From: wine-devel-bounces@winehq.org [mailto:wine-devel-bounces@winehq.org] On Behalf Of Steven Edwards Sent: 13. januar 2006 17:01 To: Rolf Kalbermatter Cc: wine-devel@winehq.org; Marcus Meissner Subject: Re: pch support
Hi,
On 1/13/06, Rolf Kalbermatter rolf.kalbermatter@citeng.com wrote:
This solution has been starting to form in my mind as well. It is even portable to MSVC it seems with a few small modifications to
winapi/msvcmaker.
I came up with a much more generic solution that will work on all compilers and does not cause a dependancy issue. It requires some minor code adjustments and there is a slight issue if you have errors in linking. The error location is wrong so you need to have a way to disable this if you have a real error to track the problem down, but once I fixed ReactOS shell32.dll to use my hack my compile times on gcc4 went from 30+ secs down to 9sec. MSVC went from 22 to less than 1 secound.
In short each module would have its own autogenerated master C file that would include all of the others like the one I have attached. Minus the #define hacks. I personally think its more of a nasty hack but a configure switch could be added like
--enable-compilation-units
Because each header is only processed once, it give the same effect as PCH.
-- Steven Edwards - ReactOS and Wine developer
"There is one thing stronger than all the armies in the world, and that is an idea whose time has come." - Victor Hugo
Hello Steven,
Alexandre did not like the needed changes required to add support for this but I figured I would throw it out for debate anyway in case anyone else can make him change his mind (duck). The issue is he does not like needing a single header per-program or per dll. After doing some research it seems to me that even MSVC does not support multiple PCH files per project so I don't know how it could be fixed in gcc to do it Microsoft has not figured it out.
Why don't you create only one single precompiled header file for <windows.h> ? The speedup effect will not that big as in your example. But the code changes are minimal. The <windows.h> include is used nearly everywhere, and it contains a great bunch of declarations. So it can be used for most of all programs and DLLs. Perhaps this change is more easy to sell.
Regards,
martin
-
Casper Hornstrup -
David Laight -
Marcus Meissner -
Martin Fuchs -
Peter Åstrand -
Rolf Kalbermatter -
Steven Edwards -
Thomas Weidenmueller