Re: [resend] Allow use of MAP_TRYFIXED for better mmap()
Todd Vierling <tv(a)pobox.com> writes:
I've implemented the patch as forward-looking, allowing other platforms to add MAP_TRYFIXED to gain the same benefit. (This mmap flag name does not appear to be used in any divergent fashion on any other platform, per my research when picking the flag's name.)
If you want to keep compatibility with older kernels, you'll need to check if MAP_TRYFIXED is available at runtime, you can't test for that at compile time. Or if you don't need to maintain compatibility then you should get rid of the NetBSD ifdefs completely. -- Alexandre Julliard julliard(a)winehq.com
On Tue, 26 Aug 2003, Alexandre Julliard wrote: : If you want to keep compatibility with older kernels, you'll need to : check if MAP_TRYFIXED is available at runtime, NetBSD isn't like Linux. In NetBSD, the kernel, includes, and libc are not bidirectionally interchangeable; they are unidirectionally upgraded. A binary compiled with newer libc and kernel is documented not to work on older kernels and libc. This is a little less true for release branches, but features do still appear between minor releases, with this same caveat. In general, this holds true with other applications on NetBSD. If a feature appears in the C headers, it is assumed to exist. The patch is in line with NetBSD's compatibility principles. Adding runtime detection code is IMHO just introducing an unnecessary OS dependency garbage code block, and I'd rather avoid that pollution. : Or if you don't need to maintain compatibility then you should get rid of : the NetBSD ifdefs completely. For now, I'd like to keep try_mmap_fixed() around for with systems that have not upgraded to kernels capable of MAP_TRYFIXED. The try_mmap_fixed() method works fine on newer NetBSD systems, so binaries compiled with older headers still work even if the kernel supports MAP_TRYFIXED. (They'll just be slower. 8-) -- -- Todd Vierling <tv(a)pobox.com>
Todd Vierling <tv(a)pobox.com> writes:
In general, this holds true with other applications on NetBSD. If a feature appears in the C headers, it is assumed to exist. The patch is in line with NetBSD's compatibility principles.
Of course this implies that everybody builds all their apps from source. But if that's what NetBSD people expect then it's OK with me. -- Alexandre Julliard julliard(a)winehq.com
On Wed, 27 Aug 2003, Alexandre Julliard wrote: : > In general, this holds true with other applications on NetBSD. If a feature : > appears in the C headers, it is assumed to exist. The patch is in line with : > NetBSD's compatibility principles. : : Of course this implies that everybody builds all their apps from : source. No, it implies that a built binary requires a kernel at least as new as the .h files used when building. This is NetBSD (and SysV and ...) standard practice, and is the foundation of how autoconf tests work. The common practice of Linux users shuffling back and forth on kernel versions is a glaring exception to this common-sense rule. 8-) -- -- Todd Vierling <tv(a)pobox.com>
Todd Vierling <tv(a)pobox.com> writes:
No, it implies that a built binary requires a kernel at least as new as the .h files used when building. This is NetBSD (and SysV and ...) standard practice, and is the foundation of how autoconf tests work.
The common practice of Linux users shuffling back and forth on kernel versions is a glaring exception to this common-sense rule. 8-)
It's not really a question of shuffling back and forth, it's that different users have different kernels; so if you want to ship a binary that works for everybody, with your method you have to stick to the lowest common denominator. With run-time checks you can have a binary that takes advantage of new kernel features, while still running everywhere. Obviously it's a bit more complex to implement, so you only want to do that where there's a real gain in using the new features; still I think the concept applies for all platforms. -- Alexandre Julliard julliard(a)winehq.com
On Wed, 27 Aug 2003, Alexandre Julliard wrote: : It's not really a question of shuffling back and forth, it's that : different users have different kernels; so if you want to ship a : binary that works for everybody, with your method you have to stick to : the lowest common denominator. With run-time checks you can have a : binary that takes advantage of new kernel features, while still : running everywhere. That's why there is a "#ifdef MAP_TRYFIXED". If this is available at compile time, it is known to be available at runtime regardless of CPU platform -- it's not an optional kernel feature. (Besides all this, detecting this particular feature at runtime is ... difficult at best. The "protected" memory range is different depending on the CPU platform.) -- -- Todd Vierling <tv(a)pobox.com>
On Wed, Aug 27, 2003 at 07:30:32PM -0400, Todd Vierling wrote:
On Wed, 27 Aug 2003, Alexandre Julliard wrote:
: It's not really a question of shuffling back and forth, it's that : different users have different kernels; so if you want to ship a : binary that works for everybody, with your method you have to stick to : the lowest common denominator. With run-time checks you can have a : binary that takes advantage of new kernel features, while still : running everywhere.
That's why there is a "#ifdef MAP_TRYFIXED". If this is available at compile time, it is known to be available at runtime regardless of CPU platform -- it's not an optional kernel feature.
It's available on the machine that the binary is compiled on sure, but what about the machine of a friend/customer who you've shipped the binary to? Just because you have the new kernel feature on your build machine doesn't mean that everybody else does. If you test at run-time you avoid these problems. Huw. -- Huw Davies huw(a)codeweavers.com
Huw D M Davies wrote:
It's available on the machine that the binary is compiled on sure, but what about the machine of a friend/customer who you've shipped the binary to? Just because you have the new kernel feature on your build machine doesn't mean that everybody else does. If you test at run-time you avoid these problems.
I believe Todd's point is that the build he's compiling won't work with any libc/kernel combination older than the one he's building with *anyway*. So for NetBSD, there'll be a build built on an ancient machine, which will work pretty much everywhere and a build built on a newish machine, which will have this define. If <random-NetBSD-Box> can run Todd's binary *at all*, then it's also guaranteed to have the facility his define's checking for. -- Jon Bright Silicon Circus Ltd. http://www.siliconcircus.com
On Thu, 28 Aug 2003, Huw D M Davies wrote: : It's available on the machine that the binary is compiled on sure, but : what about the machine of a friend/customer who you've shipped the : binary to? Just because you have the new kernel feature on your build : machine doesn't mean that everybody else does. If you test at : run-time you avoid these problems. Again, this isn't Linux we're talking about. When compiling binaries for most Unix flavors, you state a minimum requirement for running the binary when distributing, based on the version of the OS used to do the compile. The same goes, say, with features introduced in libc, such as a new function or struct layout with a newly versioned function call. Binaries are only guaranteed to run on an OS at least as new as the compilation environment. (Take Solaris, for instance. If you compile a binary on Solaris 2.5.1, it will likely run on 2.6, 7, 8, and 9, but is not guaranteed to work on 2.5, 2.4, or earlier releases.) I have a feeling we're just on different mental wavelengths here. MAP_TRYFIXED is not an optional feature. If the #define exists, the feature exists -- for the OS version used to compile, and all later versions. -- -- Todd Vierling <tv(a)pobox.com>
Todd Vierling <tv(a)pobox.com> writes:
I have a feeling we're just on different mental wavelengths here. MAP_TRYFIXED is not an optional feature. If the #define exists, the feature exists -- for the OS version used to compile, and all later versions.
You are still thinking only about source distributions. The problem is when you want to ship a binary: the only way to build a binary that works for everybody is to build it on an old OS version; that's the way it is on all systems (it's the same way on Linux, at least WRT libc). Then, since you built that binary on an old OS, it will not use MAP_TRYFIXED, no matter what kernel it runs on. Basically you have to sacrifice performance to gain portability; even if 99% of your users have a new kernel, your binary still cannot take advantage of that. With a run-time check you don't have to sacrifice anything, the binary is still portable everywhere, but can also use MAP_TRYFIXED for better performance if it exists in the running kernel. -- Alexandre Julliard julliard(a)winehq.com
On Thu, 28 Aug 2003, Alexandre Julliard wrote: : > I have a feeling we're just on different mental wavelengths here. : > MAP_TRYFIXED is not an optional feature. If the #define exists, the feature : > exists -- for the OS version used to compile, and all later versions. : : You are still thinking only about source distributions. The problem is : when you want to ship a binary: the only way to build a binary that : works for everybody is to build it on an old OS version; Yes, that's right. This is not a problem, as 99% of Wine users on NetBSD are building from source. The very small remainder of prebuilt-binary users can easily be told to ensure that the OS version is up-to-date (since there are security fixes that necessitate this anyway). : With a run-time check you don't have to sacrifice anything, You have to bend over backwards to do the runtime check, and you sacrifice *maintainability* with the extra normally-dead code added to Wine. I don't see a need to jump through extra hoops in this case. -- -- Todd Vierling <tv(a)pobox.com>
Todd Vierling <tv(a)pobox.com> writes:
You have to bend over backwards to do the runtime check, and you sacrifice *maintainability* with the extra normally-dead code added to Wine. I don't see a need to jump through extra hoops in this case.
I think that run-time checks are more maintainable that compile-time checks, since it means the code gets exercised on all machines, so if you break it it gets noticed at once. If you break something that's inside a rarely used #ifdef block you may not find the problem until 6 months later. It's also a lot easier to follow the code if you don't have to wonder which parts of it are actually compiled in. Anyway it doesn't really matter for that specific issue, if you don't feel the performance gain is worth the trouble that's your call. I was just raising a general point that checking run-time features with #ifdef is not a good idea, no matter how much autoconf encourages that style. -- Alexandre Julliard julliard(a)winehq.com
On Thu, 28 Aug 2003, Alexandre Julliard wrote: : Anyway it doesn't really matter for that specific issue, if you don't : feel the performance gain is worth the trouble that's your call. It's not. NetBSD's userbase [capable of running Wine] is moving rapidly enough, and enough are compiling Wine from source, that this is all a non-issue and the simple few-line diff I supplied is the easiest. : I was just raising a general point that checking run-time features with : #ifdef is not a good idea, no matter how much autoconf encourages that : style. But it is, depending on what you need to do. Duplicating #defines, struct declarations, etc. from newer OS versions is prone to error -- not to mention heavily OS dependent! -- and does not let the OS provide its proper degree of source compatibility that is common to all Unix flavors, including Linux. That's why autoconf has tests, and why people have used #ifdef for decades. (Autoconf is big and bloated, to be sure, but there's a reason it is so damned popular.) I definitely don't want to reproduce apt-to-change parts of the OS in an application just because it *might* be built on an earlier version of that OS. If an app is built on the earlier OS version, the user should expect to be missing features. Unix users accept this as a fact of life. This is not to say that there is no case where forward-looking checks are useful. The Linux VFAT ioctls are a marginally good example where such a check can be done with low impact, but these cases should be evaluated carefully and kept in the "rare" category as much as possible. -- -- Todd Vierling <tv(a)pobox.com>
Todd Vierling <tv(a)pobox.com> writes:
I definitely don't want to reproduce apt-to-change parts of the OS in an application just because it *might* be built on an earlier version of that OS. If an app is built on the earlier OS version, the user should expect to be missing features. Unix users accept this as a fact of life.
But like it or not, this is changing. Maybe not for NetBSD yet, but there are many people at least on Linux who are not primarily Unix users, and who want things to work without having to mess with configure or upgrading system headers. This is doubly true for Wine, since "real" Unix users don't need Windows apps, our primary audience is Windows users who are used to downloading binaries that "just work". And so we have a responsibility to make sure that it's the case for them, even if it means departing from the One True Unix Way. -- Alexandre Julliard julliard(a)winehq.com
--- Todd Vierling <tv(a)pobox.com> wrote:
On Thu, 28 Aug 2003, Alexandre Julliard wrote:
: > I have a feeling we're just on different mental wavelengths here. : > MAP_TRYFIXED is not an optional feature. If the #define exists, the feature : > exists -- for the OS version used to compile, and all later versions. : : You are still thinking only about source distributions. The problem is : when you want to ship a binary: the only way to build a binary that : works for everybody is to build it on an old OS version;
Yes, that's right. This is not a problem, as 99% of Wine users on NetBSD are building from source. The very small remainder of prebuilt-binary users can easily be told to ensure that the OS version is up-to-date (since there are security fixes that necessitate this anyway).
: With a run-time check you don't have to sacrifice anything,
You have to bend over backwards to do the runtime check, and you sacrifice *maintainability* with the extra normally-dead code added to Wine. I don't see a need to jump through extra hoops in this case.
Shouldnt something like this added to the loader work for a runtime check? (note im not using proper function names or anything, just an example) if (running_on_netbsd && version >= first_version_that_supports_MAP_TRYFIXED) #define can_use_MAP_TRYFIXED Im no expert on NetBSD but I would think a simple check like that would suffice, and since it is in the loader, it would make it a runtime check, no? ===== -- Dustin Navea Minor Contributor, http://www.winehq.com Bugzilla Janitor, http://bugs.winehq.com Network Admin, irc://irc.blynk.net (down) __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com
On Fri, 29 Aug 2003, Dustin Navea wrote: : Shouldnt something like this added to the loader work for a runtime check? : (note im not using proper function names or anything, just an example) : : if (running_on_netbsd && version >= first_version_that_supports_MAP_TRYFIXED) : #define can_use_MAP_TRYFIXED That kind of OS dependency runtime check does not belong in an application that runs on multiple OS's. In every sane application, both "running_on_netbsd" AND "version" are constants, so the test is either always true or always false. What it comes down to is this: NetBSD doesn't do all the kernel and libc shuffling of Linux. Its version numbers are quite well defined, and it's trivial to state what version was used to compile a binary. So the patch is Just Fine as-is, and needs no further pollution. -- -- Todd Vierling <tv(a)pobox.com>
Alexandre Julliard wrote:
With a run-time check you don't have to sacrifice anything, the binary is still portable everywhere, but can also use MAP_TRYFIXED for better performance if it exists in the running kernel.
Except that if you build it on a machine which knows about MAP_TRYFIXED at all, the resulting binary's not likely to run on older machines... -- Jon Bright Silicon Circus Ltd. http://www.siliconcircus.com
On Thu, 28 Aug 2003, Jon Bright wrote:
Except that if you build it on a machine which knows about MAP_TRYFIXED at all, the resulting binary's not likely to run on older machines...
No, if the check is done at runtime, there's no problem. Where do you see any? -- Dimi.
Dimitrie O. Paun wrote:
On Thu, 28 Aug 2003, Jon Bright wrote:
Except that if you build it on a machine which knows about MAP_TRYFIXED at all, the resulting binary's not likely to run on older machines...
No, if the check is done at runtime, there's no problem. Where do you see any?
Having the necessary headers to compile with the macros/functions/whatever for MAP_TRYFIXED implies an up-to-date libc. An up-to-date libc implies the resulting binary will not run on older machines. -- Jon Bright Silicon Circus Ltd. http://www.siliconcircus.com
Jon Bright <jon(a)siliconcircus.com> writes:
Except that if you build it on a machine which knows about MAP_TRYFIXED at all, the resulting binary's not likely to run on older machines...
You don't need the machine to know about it, that's the whole point. Of course that means you need to define the symbol yourself if it's not defined already. -- Alexandre Julliard julliard(a)winehq.com
On Thu, Aug 28, 2003 at 07:07:25PM +0200, Jon Bright wrote:
Alexandre Julliard wrote:
With a run-time check you don't have to sacrifice anything, the binary is still portable everywhere, but can also use MAP_TRYFIXED for better performance if it exists in the running kernel.
Except that if you build it on a machine which knows about MAP_TRYFIXED at all, the resulting binary's not likely to run on older machines...
It is exteremely unlikely to run anyway due to changes in other parts of the ABI. For instance the dynamic linker in NetBSD 1.5.1 cannot load programs compiled under NetBSD current (NetBSD current can run programs compiled under 1.5.1). It is called 'backwards compatibility'..... David -- David Laight: dsl(a)netbsd.org
participants (7)
-
Alexandre Julliard -
David Laight -
Dimitrie O. Paun -
Dustin Navea -
Huw D M Davies -
Jon Bright -
Todd Vierling