RE: wineinstall - Add support for parallel builds using 'nproc' to detect cpu count
Hi, i like/use "make -j 1" for log comparison. Can you leave NPROC=1 or allow override? W.
On Fri, Feb 24, 2017 at 2:05 AM, <wylda(a)volny.cz> wrote:
Hi, i like/use "make -j 1" for log comparison. Can you leave NPROC=1 or allow override?
W.
Hi Wylda, Are you doing that with wineinstall? This doesn't affect normal ./configure && make, only ./tools/wineinstall. That said, allowing the user to set it via NPROC or similar would be a good idea IMO. -- -Austin GPG: 14FB D7EA A041 937B
On Fri, Feb 24, 2017 at 3:11 AM, Austin English <austinenglish(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 2:05 AM, <wylda(a)volny.cz> wrote:
Hi, i like/use "make -j 1" for log comparison. Can you leave NPROC=1 or allow override?
W.
Hi Wylda,
Are you doing that with wineinstall? This doesn't affect normal ./configure && make, only ./tools/wineinstall.
That said, allowing the user to set it via NPROC or similar would be a good idea IMO.
-- -Austin GPG: 14FB D7EA A041 937B
Is this what you had in mind? I wasn't sure how to document that you could do: NPROC=50 ./tools/wineinstall to force 50 parallel processes. commit 83321d6452706b5f8f93687ed4e0909aac47fd68 Author: Chris Morgan <chmorgan(a)gmail.com> Date: Tue Feb 21 17:03:30 2017 -0500 wineinstall - Add support for parallel builds using 'nproc' to detect cpu count Parallel builds can greatly reduce the overall build time on modern multi-core processors. Fall back to two parallel builds in the case where nproc is unavailable, most modern processors have at least two cores. Use the 'NPROC' value defined by the environment if it is not null. diff --git a/tools/wineinstall b/tools/wineinstall index e8e22bf..a757343 100755 --- a/tools/wineinstall +++ b/tools/wineinstall @@ -143,10 +143,22 @@ echo "in the meantime..." echo std_sleep +# determine ideal number of parallel processes if NPROC isn't set +# in the environment +if [ ! -n $NPROC ] +then + if [ -x `which nproc 2>/dev/null` ] + then + NPROC=$(nproc --all) + else + NPROC=2 + fi +fi + # try to just make wine, if this fails 'make depend' and try to remake -if ! { make; } +if ! { make -j$NPROC; } then - if ! { make depend && make; } + if ! { make depend && make -j$NPROC; } then echo echo "Compilation failed, aborting install."
On Fri, Feb 24, 2017 at 6:53 AM, Chris Morgan <chmorgan(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 3:11 AM, Austin English <austinenglish(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 2:05 AM, <wylda(a)volny.cz> wrote:
Hi, i like/use "make -j 1" for log comparison. Can you leave NPROC=1 or allow override?
W.
Hi Wylda,
Are you doing that with wineinstall? This doesn't affect normal ./configure && make, only ./tools/wineinstall.
That said, allowing the user to set it via NPROC or similar would be a good idea IMO.
-- -Austin GPG: 14FB D7EA A041 937B
Is this what you had in mind? I wasn't sure how to document that you could do:
NPROC=50 ./tools/wineinstall
to force 50 parallel processes.
commit 83321d6452706b5f8f93687ed4e0909aac47fd68 Author: Chris Morgan <chmorgan(a)gmail.com> Date: Tue Feb 21 17:03:30 2017 -0500
wineinstall - Add support for parallel builds using 'nproc' to detect cpu count
Parallel builds can greatly reduce the overall build time on modern multi-core processors.
Fall back to two parallel builds in the case where nproc is unavailable, most modern processors have at least two cores.
Use the 'NPROC' value defined by the environment if it is not null.
diff --git a/tools/wineinstall b/tools/wineinstall index e8e22bf..a757343 100755 --- a/tools/wineinstall +++ b/tools/wineinstall @@ -143,10 +143,22 @@ echo "in the meantime..." echo std_sleep
+# determine ideal number of parallel processes if NPROC isn't set +# in the environment +if [ ! -n $NPROC ] +then + if [ -x `which nproc 2>/dev/null` ] + then + NPROC=$(nproc --all) + else + NPROC=2 + fi +fi + # try to just make wine, if this fails 'make depend' and try to remake -if ! { make; } +if ! { make -j$NPROC; } then - if ! { make depend && make; } + if ! { make depend && make -j$NPROC; } then echo echo "Compilation failed, aborting install."
Yeah, something like that (but please avoid the backticks). -- -Austin GPG: 14FB D7EA A041 937B
On Fri, Feb 24, 2017 at 1:24 PM Austin English <austinenglish(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 6:53 AM, Chris Morgan <chmorgan(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 3:11 AM, Austin English <austinenglish(a)gmail.com
wrote:
On Fri, Feb 24, 2017 at 2:05 AM, <wylda(a)volny.cz> wrote:
Hi, i like/use "make -j 1" for log comparison. Can you leave NPROC=1 or allow override?
W.
Hi Wylda,
Are you doing that with wineinstall? This doesn't affect normal ./configure && make, only ./tools/wineinstall.
That said, allowing the user to set it via NPROC or similar would be a good idea IMO.
-- -Austin GPG: 14FB D7EA A041 937B
Is this what you had in mind? I wasn't sure how to document that you could do:
NPROC=50 ./tools/wineinstall
to force 50 parallel processes.
commit 83321d6452706b5f8f93687ed4e0909aac47fd68 Author: Chris Morgan <chmorgan(a)gmail.com> Date: Tue Feb 21 17:03:30 2017 -0500
wineinstall - Add support for parallel builds using 'nproc' to detect cpu count
Parallel builds can greatly reduce the overall build time on modern multi-core processors.
Fall back to two parallel builds in the case where nproc is unavailable, most modern processors have at least two cores.
Use the 'NPROC' value defined by the environment if it is not null.
diff --git a/tools/wineinstall b/tools/wineinstall index e8e22bf..a757343 100755 --- a/tools/wineinstall +++ b/tools/wineinstall @@ -143,10 +143,22 @@ echo "in the meantime..." echo std_sleep
+# determine ideal number of parallel processes if NPROC isn't set +# in the environment +if [ ! -n $NPROC ] +then + if [ -x `which nproc 2>/dev/null` ] + then + NPROC=$(nproc --all) + else + NPROC=2 + fi +fi + # try to just make wine, if this fails 'make depend' and try to remake -if ! { make; } +if ! { make -j$NPROC; } then - if ! { make depend && make; } + if ! { make depend && make -j$NPROC; } then echo echo "Compilation failed, aborting install."
Yeah, something like that (but please avoid the backticks).
-- -Austin GPG: 14FB D7EA A041 937B
Hi Austin. Can you clarify what you mean and comment on the documentation for how to override nproc with the environment variable? I'd like to update and send out a new patch. Chris
On Fri, Feb 24, 2017 at 12:26 PM, Chris Morgan <chmorgan(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 1:24 PM Austin English <austinenglish(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 6:53 AM, Chris Morgan <chmorgan(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 3:11 AM, Austin English <austinenglish(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 2:05 AM, <wylda(a)volny.cz> wrote:
Hi, i like/use "make -j 1" for log comparison. Can you leave NPROC=1 or allow override?
W.
Hi Wylda,
Are you doing that with wineinstall? This doesn't affect normal ./configure && make, only ./tools/wineinstall.
That said, allowing the user to set it via NPROC or similar would be a good idea IMO.
-- -Austin GPG: 14FB D7EA A041 937B
Is this what you had in mind? I wasn't sure how to document that you could do:
NPROC=50 ./tools/wineinstall
to force 50 parallel processes.
commit 83321d6452706b5f8f93687ed4e0909aac47fd68 Author: Chris Morgan <chmorgan(a)gmail.com> Date: Tue Feb 21 17:03:30 2017 -0500
wineinstall - Add support for parallel builds using 'nproc' to detect cpu count
Parallel builds can greatly reduce the overall build time on modern multi-core processors.
Fall back to two parallel builds in the case where nproc is unavailable, most modern processors have at least two cores.
Use the 'NPROC' value defined by the environment if it is not null.
diff --git a/tools/wineinstall b/tools/wineinstall index e8e22bf..a757343 100755 --- a/tools/wineinstall +++ b/tools/wineinstall @@ -143,10 +143,22 @@ echo "in the meantime..." echo std_sleep
+# determine ideal number of parallel processes if NPROC isn't set +# in the environment +if [ ! -n $NPROC ] +then + if [ -x `which nproc 2>/dev/null` ] + then + NPROC=$(nproc --all) + else + NPROC=2 + fi +fi + # try to just make wine, if this fails 'make depend' and try to remake -if ! { make; } +if ! { make -j$NPROC; } then - if ! { make depend && make; } + if ! { make depend && make -j$NPROC; } then echo echo "Compilation failed, aborting install."
Yeah, something like that (but please avoid the backticks).
-- -Austin GPG: 14FB D7EA A041 937B
Hi Austin.
Can you clarify what you mean and comment on the documentation for how to override nproc with the environment variable? I'd like to update and send out a new patch.
Chris
What you sent should work. Since nproc isn't always available, there's not a super clean way to do it, but a slightly cleaner alternative would be: if [ -x $(which nproc) ] ; then NPROC="${NPROC:-$(nproc --all)}" else NPROC="${NPROC:-2}" fi -- -Austin GPG: 14FB D7EA A041 937B
On Fri, Feb 24, 2017 at 1:32 PM Austin English <austinenglish(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 12:26 PM, Chris Morgan <chmorgan(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 1:24 PM Austin English <austinenglish(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 6:53 AM, Chris Morgan <chmorgan(a)gmail.com>
wrote:
On Fri, Feb 24, 2017 at 3:11 AM, Austin English <austinenglish(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 2:05 AM, <wylda(a)volny.cz> wrote:
Hi, i like/use "make -j 1" for log comparison. Can you leave NPROC=1 or allow override?
W.
Hi Wylda,
Are you doing that with wineinstall? This doesn't affect normal ./configure && make, only ./tools/wineinstall.
That said, allowing the user to set it via NPROC or similar would be
a
good idea IMO.
-- -Austin GPG: 14FB D7EA A041 937B
Is this what you had in mind? I wasn't sure how to document that you could do:
NPROC=50 ./tools/wineinstall
to force 50 parallel processes.
commit 83321d6452706b5f8f93687ed4e0909aac47fd68 Author: Chris Morgan <chmorgan(a)gmail.com> Date: Tue Feb 21 17:03:30 2017 -0500
wineinstall - Add support for parallel builds using 'nproc' to detect cpu count
Parallel builds can greatly reduce the overall build time on modern multi-core processors.
Fall back to two parallel builds in the case where nproc is unavailable, most modern processors have at least two cores.
Use the 'NPROC' value defined by the environment if it is not null.
diff --git a/tools/wineinstall b/tools/wineinstall index e8e22bf..a757343 100755 --- a/tools/wineinstall +++ b/tools/wineinstall @@ -143,10 +143,22 @@ echo "in the meantime..." echo std_sleep
+# determine ideal number of parallel processes if NPROC isn't set +# in the environment +if [ ! -n $NPROC ] +then + if [ -x `which nproc 2>/dev/null` ] + then + NPROC=$(nproc --all) + else + NPROC=2 + fi +fi + # try to just make wine, if this fails 'make depend' and try to remake -if ! { make; } +if ! { make -j$NPROC; } then - if ! { make depend && make; } + if ! { make depend && make -j$NPROC; } then echo echo "Compilation failed, aborting install."
Yeah, something like that (but please avoid the backticks).
-- -Austin GPG: 14FB D7EA A041 937B
Hi Austin.
Can you clarify what you mean and comment on the documentation for how to override nproc with the environment variable? I'd like to update and send out a new patch.
Chris
What you sent should work. Since nproc isn't always available, there's not a super clean way to do it, but a slightly cleaner alternative would be:
if [ -x $(which nproc) ] ; then NPROC="${NPROC:-$(nproc --all)}" else NPROC="${NPROC:-2}" fi
-- -Austin GPG: 14FB D7EA A041 937B
Ahh, that is simpler. I saw the conditional style when refreshing on tests and environment variables passed into scripts. I also read that using type was preferred over which on a so post. I'm ok with either way though. I'll resend an updated patch. Chris
On Fri, Feb 24, 2017 at 12:35 PM, Chris Morgan <chmorgan(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 1:32 PM Austin English <austinenglish(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 12:26 PM, Chris Morgan <chmorgan(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 1:24 PM Austin English <austinenglish(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 6:53 AM, Chris Morgan <chmorgan(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 3:11 AM, Austin English <austinenglish(a)gmail.com> wrote:
On Fri, Feb 24, 2017 at 2:05 AM, <wylda(a)volny.cz> wrote: > Hi, > i like/use "make -j 1" for log comparison. Can you leave NPROC=1 > or > allow override? > > W. > >
Hi Wylda,
Are you doing that with wineinstall? This doesn't affect normal ./configure && make, only ./tools/wineinstall.
That said, allowing the user to set it via NPROC or similar would be a good idea IMO.
-- -Austin GPG: 14FB D7EA A041 937B
Is this what you had in mind? I wasn't sure how to document that you could do:
NPROC=50 ./tools/wineinstall
to force 50 parallel processes.
commit 83321d6452706b5f8f93687ed4e0909aac47fd68 Author: Chris Morgan <chmorgan(a)gmail.com> Date: Tue Feb 21 17:03:30 2017 -0500
wineinstall - Add support for parallel builds using 'nproc' to detect cpu count
Parallel builds can greatly reduce the overall build time on modern multi-core processors.
Fall back to two parallel builds in the case where nproc is unavailable, most modern processors have at least two cores.
Use the 'NPROC' value defined by the environment if it is not null.
diff --git a/tools/wineinstall b/tools/wineinstall index e8e22bf..a757343 100755 --- a/tools/wineinstall +++ b/tools/wineinstall @@ -143,10 +143,22 @@ echo "in the meantime..." echo std_sleep
+# determine ideal number of parallel processes if NPROC isn't set +# in the environment +if [ ! -n $NPROC ] +then + if [ -x `which nproc 2>/dev/null` ] + then + NPROC=$(nproc --all) + else + NPROC=2 + fi +fi + # try to just make wine, if this fails 'make depend' and try to remake -if ! { make; } +if ! { make -j$NPROC; } then - if ! { make depend && make; } + if ! { make depend && make -j$NPROC; } then echo echo "Compilation failed, aborting install."
Yeah, something like that (but please avoid the backticks).
-- -Austin GPG: 14FB D7EA A041 937B
Hi Austin.
Can you clarify what you mean and comment on the documentation for how to override nproc with the environment variable? I'd like to update and send out a new patch.
Chris
What you sent should work. Since nproc isn't always available, there's not a super clean way to do it, but a slightly cleaner alternative would be:
if [ -x $(which nproc) ] ; then NPROC="${NPROC:-$(nproc --all)}" else NPROC="${NPROC:-2}" fi
-- -Austin GPG: 14FB D7EA A041 937B
Ahh, that is simpler. I saw the conditional style when refreshing on tests and environment variables passed into scripts.
I also read that using type was preferred over which on a so post. I'm ok with either way though. I'll resend an updated patch.
Chris
My understanding is `command -v` is the most portable, but that's somewhat debatable. Keep in mind that this only needs to support platforms that wine supports, so extremely old stuff can probably be ignored. I kept which simply because that's what the rest uses, but that's totally your call. If you're interested in the which/type/command debate, see https://unix.stackexchange.com/questions/85249/why-not-use-which-what-to-use... -- -Austin GPG: 14FB D7EA A041 937B
Austin English <austinenglish(a)gmail.com> writes:
Yeah, something like that (but please avoid the backticks).
Actually we use backticks everywhere because that was necessary for Solaris. -- Alexandre Julliard julliard(a)winehq.org
On Fri, Feb 24, 2017 at 12:51 PM, Alexandre Julliard <julliard(a)winehq.org> wrote:
Austin English <austinenglish(a)gmail.com> writes:
Yeah, something like that (but please avoid the backticks).
Actually we use backticks everywhere because that was necessary for Solaris.
-- Alexandre Julliard julliard(a)winehq.org
Seriously? Is that still the case / what version of Solaris was that, if you remember? -- -Austin GPG: 14FB D7EA A041 937B
Austin English <austinenglish(a)gmail.com> writes:
On Fri, Feb 24, 2017 at 12:51 PM, Alexandre Julliard <julliard(a)winehq.org> wrote:
Austin English <austinenglish(a)gmail.com> writes:
Yeah, something like that (but please avoid the backticks).
Actually we use backticks everywhere because that was necessary for Solaris.
-- Alexandre Julliard julliard(a)winehq.org
Seriously? Is that still the case / what version of Solaris was that, if you remember?
Solaris seems to never upgrade the core utilities, so any version I'd guess. But I haven't touched Solaris in a while (fortunately ;-) -- Alexandre Julliard julliard(a)winehq.org
On Fri, Feb 24, 2017 at 1:02 PM, Alexandre Julliard <julliard(a)winehq.org> wrote:
Austin English <austinenglish(a)gmail.com> writes:
On Fri, Feb 24, 2017 at 12:51 PM, Alexandre Julliard <julliard(a)winehq.org> wrote:
Austin English <austinenglish(a)gmail.com> writes:
Yeah, something like that (but please avoid the backticks).
Actually we use backticks everywhere because that was necessary for Solaris.
-- Alexandre Julliard julliard(a)winehq.org
Seriously? Is that still the case / what version of Solaris was that, if you remember?
Solaris seems to never upgrade the core utilities, so any version I'd guess. But I haven't touched Solaris in a while (fortunately ;-)
-- Alexandre Julliard julliard(a)winehq.org
Smart ;). Francois, do you know if this still an issue? -- -Austin GPG: 14FB D7EA A041 937B
On Fri, 24 Feb 2017, Austin English wrote: [...]
Actually we use backticks everywhere because that was necessary for Solaris.
Seriously? Is that still the case / what version of Solaris was that, if you remember?
Solaris seems to never upgrade the core utilities, so any version I'd guess. But I haven't touched Solaris in a while (fortunately ;-)
Francois, do you know if this still an issue?
I don't run Solaris much either. Still have my Solaris 11.11 VM though. I ran the following script on it: $ cat >foo <<EOF b=`which nproc` echo "b=[$b]" p=$(which nproc) echo "p=[$p]" EOF $ /bin/sh foo b=[/usr/bin/nproc] p=[/usr/bin/nproc] $ /bin/sh --version | head -n 1 GNU bash, version 4.1.9(1)-release (i386-pc-solaris2.11) $ /bin/ksh foo b=[/usr/bin/nproc] p=[/usr/bin/nproc] $ /bin/ksh --version version sh (AT&T Research) 93u 2011-02-08 So it seems to work. That said if the rest of the code does it one way and the new way does not bring a significant improvement I see no reason to stray from existing practices. -- Francois Gouget <fgouget(a)free.fr> http://fgouget.free.fr/ You can have my guns when you pry them from my kids cold, dead hands.
participants (5)
-
Alexandre Julliard -
Austin English -
Chris Morgan -
Francois Gouget -
wylda＠volny.cz