42

I can set number of threads for the build process using -j argument. For example, I have 4 cores +4 virtual. When I write: make -j8 the speed increases 4 times.

Is it possible to set that value as default? (For example, in Linux Gentoo, in config file, it's possible to set this default value).

p.s. I have Arch Linux

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Max Frai
  • 61,946
  • 78
  • 197
  • 306
  • So.. the default C++ make option is 'run 4 times slower than is possible'? What a strange decision! – Martin James May 12 '12 at 22:36
  • 1
    @MartinJames Parallel make has many flaws and so it makes a bad default. – Pubby May 12 '12 at 22:39
  • 7
    Pubby: Most makefile authors have numerous flaws and don't get the dependencies right, ultimately leading to bad experiences when trying parallel builds ;) – 0xC0000022L May 12 '12 at 23:02
  • 1
    But often, just rerunning `make` once more makes everything right while still having built most of the stuff much faster. – K3---rnc Mar 07 '17 at 00:16
  • @K3---rnc I have a question that illustrates what you have said: [Error building wxWidgets using -j (jobs) option with MinGW](https://stackoverflow.com/q/29424990/3258851) – Marc.2377 Aug 08 '17 at 01:52
  • Does this answer your question? [Make "make" default to "make -j 8"](https://stackoverflow.com/questions/2151605/make-make-default-to-make-j-8) – user202729 May 19 '23 at 03:14

3 Answers3

44

Your question is not about threads, but processes (jobs) executed by make.

The simple, way to set this, when make is used from the console is adding:

alias make="/usr/bin/make -j 8"

to your .profile file.

You can also use setenv MAKEFLAGS '-j 8', but MAKEFLAGS can ignore this parameter in some scenarios, because keeping desired number of processes requires communicating with recursive make calls. Happily this method works with current versions of GNU Make.

Rafał Rawicki
  • 22,324
  • 5
  • 59
  • 79
  • 9
    Since this mentioning of "recursive make" doesn't stop, I'd like to once again point to http://aegis.sourceforge.net/auug97.pdf ... besides, the make file needs to be well-written to even allow for parallel make. Often, especially automated methods, disregard the actual dependencies and cause targets to be built multiple times and the likes ... – 0xC0000022L May 12 '12 at 23:01
  • 2
    In case the link is not working in future, the paper is titled "Recursive Make Considered Harmful" and should be easy enough to find. It is well worth a read. – SystemParadox Mar 30 '17 at 21:40
  • What's `setenv` here? The csh builtin? – alecov Nov 27 '17 at 21:00
  • 1
    Most people should NOT pass the `-j` option value down to recursive calls. Jobserver in the top-level `make` does this much better: all the child processes combined get exactly as many jobs as the cores assigned. Now imagine if you set -j16 for 16 cores, and EACH of 10 child `make`s gets `-j16` (which overrides jobserver). Now you have 160 jobs doing context switches on a CPU-bound tasks. – Victor Sergienko Nov 29 '20 at 20:50
37
setenv MAKEFLAGS '-j8'

Hope this helps!

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Matt Melton
  • 2,493
  • 19
  • 25
  • 3
    Note that `setenv` belongs to `(t)csh` and is not available in `bash` (see https://unix.stackexchange.com/a/85100/200009). – Patrick Sep 07 '20 at 15:05
22

Here's how I've done it:

CORES ?= $(shell sysctl -n hw.ncpu || echo 1)

all:; @$(MAKE) _all -j$(CORES)
_all: install lint test
.PHONY: all _all
…

I've basically "aliased" my default target all to a "private" _all. The command to figure out the number of cores is OSX specific, AFAIK, so you could just improve it to be more cross platform if you will. And because of the ?= assignment, we can just override it with and env variable if/when needed.

EDIT:

You can also append to your MAKEFLAGS from within the makefile itself, like so:

CPUS ?= $(shell sysctl -n hw.ncpu || echo 1)
MAKEFLAGS += --jobs=$(CPUS)
…

EDIT 2:

You may also use the following, ff you want it to be more cross-platform:

CPUS ?= $(shell (nproc --all || sysctl -n hw.ncpu) 2>/dev/null || echo 1)
gtramontina
  • 1,096
  • 9
  • 10
  • 2
    This is IMHO better than the other solutions, since you can tailor this per Makefile. – Prof. Falken Mar 23 '17 at 14:22
  • You can't alter MAKEFLAGS this way. Try it: https://gist.github.com/sanmai/dcc31ae20afa6e8ba4721f174fe05fd9 – sanmai Feb 01 '18 at 09:01
  • @sanmai The Linux kernel Makefile (https://github.com/torvalds/linux/blob/master/Makefile) modifies MAKEFLAGS in a similar way, and I have a similar jobs modification that works for me in my project. – jbo5112 Nov 05 '19 at 00:40
  • @jbo5112 true you can do that in the current version of the make program; before there was a long time where you could not – sanmai Nov 05 '19 at 05:47
  • @sanmai I was doing it back in the late 90's with Slackware 3 and the 2.1 kernel, because I didn't know I could just pass it as an argument, and it still works on RHEL7 (Make 3.82). Must have been a bug introduced where it didn't work for a while. – jbo5112 Nov 05 '19 at 19:04
  • @jbo5112 it kinda worked like if you supplied -j without any arguments even if you did, so you were lucky not having met an accidental forkbomb. [Here's a changelog entry with the fix.](https://lists.gnu.org/archive/html/info-gnu/2016-05/msg00013.html) – sanmai Nov 06 '19 at 05:26
  • I think I like this answer. It lets me have better control when I do it. For instance, I often do "make clean all", and I would definitely not want them running in parallel. I also have rules that set up temp directories and such which should be done left-to-right, not parallel. Nice. Note, however, the sysctl won't work in a docker container, but this answer far predates that being a common thing. – Joseph Larson Jun 14 '21 at 14:46
  • @JosephLarson, I've been using `CPUS ?= $(shell (nproc --all || sysctl -n hw.ncpu) 2>/dev/null || echo 1)` since then to address what you mentioned. [maybe I'll just update the answer :-)] – gtramontina Jun 16 '21 at 04:24