283

I want to execute the following commands:

./a.out 1
./a.out 2
./a.out 3
./a.out 4
.
.
. and so on

How to write this thing as a loop in a Makefile?

Willem Van Onsem
  • 443,496
  • 30
  • 428
  • 555
avd
  • 13,993
  • 32
  • 78
  • 99

16 Answers16

360

The following will do it if, as I assume by your use of ./a.out, you're on a UNIX-type platform.

for number in 1 2 3 4 ; do \
    ./a.out $$number ; \
done

Test as follows:

target:
    for number in 1 2 3 4 ; do \
        echo $$number ; \
    done

produces:

1
2
3
4

For bigger ranges, use:

target:
    number=1 ; while [[ $$number -le 10 ]] ; do \
        echo $$number ; \
        ((number = number + 1)) ; \
    done

This outputs 1 through 10 inclusive, just change the while terminating condition from 10 to 1000 for a much larger range as indicated in your comment.

Nested loops can be done thus:

target:
    num1=1 ; while [[ $$num1 -le 4 ]] ; do \
        num2=1 ; while [[ $$num2 -le 3 ]] ; do \
            echo $$num1 $$num2 ; \
            ((num2 = num2 + 1)) ; \
        done ; \
        ((num1 = num1 + 1)) ; \
    done

producing:

1 1
1 2
1 3
2 1
2 2
2 3
3 1
3 2
3 3
4 1
4 2
4 3
John Hamilton
  • 125
  • 1
  • 9
paxdiablo
  • 854,327
  • 234
  • 1,573
  • 1,953
  • But if I want to go up to 1000, then cant we specify the limit? – avd Sep 29 '09 at 06:48
  • One thing is What is the use of the label qwert? Other thing is nested loops. It shows syntax error in this for n1 in 0 1 2 ; do \ for n2 in 1 2 3 ; do \ ./a.out $$n1 $$n2 ; \ done done – avd Sep 29 '09 at 07:01
  • 1
    qwert is just a target name that's unlikely to be a real file. Makefile rules need a target. As for the syntax error, you're missing some stuff - see update. – paxdiablo Sep 29 '09 at 07:04
  • 4
    Since you're assuming his shell recognizes ((...)), why not use the much simpler for ((i = 0; i < WHATEVER; ++i)); do ...; done ? – Idelic Sep 29 '09 at 07:13
  • 1
    Your second example doesn't work for me. make version: 3.82. Prints: `make: Nothing to be done for 'qwert'.` Plz, help. – user4035 Apr 27 '13 at 19:27
  • 4
    Note that the `seq` command which generates a sequence of numbers exists on most (all?) unix systems, so you can write `for number in ``seq 1 1000``; do echo $$number; done` (Put a single backtick on each side of the seq command, not two, I don't know how to format this correctly using stackoverflow's syntax) – Suzanne Soy Sep 23 '13 at 10:31
  • 4
    Thanks, I was missing the double $$ to reference the for loop variable. – Leif Gruenwoldt Oct 04 '13 at 14:40
  • since the syntax is the one of bash, it would be even simpler to use the built-in interval generator: {1..4} or {1..1000} – user1251840 Oct 02 '15 at 08:40
  • 1
    Note that it's important to use `$$` for the variable instead of just `$` – Ameo Sep 22 '16 at 21:59
  • i used the first case, and if i use `make qwert`, it prints also the command.. someone knows how to fix it ? – SevO Nov 30 '17 at 14:32
  • 2
    can someone explain the need for the double $$ and the line continuation character /? I'm new to makefiles but I don't see this kind of encoding anywhere else – jrz Sep 24 '19 at 19:11
  • 3
    @Jonz: the line continuation character is because `make` normally treats each line as a thing to run in a *separate* sub-shell. Without continuation, it would try to run a subshell with just (for example) `for number in 1 2 3 4 ; do`, without the rest of the loop. With it, it effectively becomes a single line of the form `while something ; do something ; done`, which is a complete statement. The `$$` question is answered here: https://stackoverflow.com/questions/26564825/what-is-the-meaning-of-a-double-dollar-sign-in-bash-makefile, with the code seemingly extracted from this very answer :-) – paxdiablo Sep 25 '19 at 00:26
  • 1
    @Jonz: Keep in mind you *could* actually have the single line `while something ; do something ; done` without continuation but that could get rather unreadable for large loops (like that nested example I gave). The continuation allows better formatting. – paxdiablo Sep 25 '19 at 00:29
  • I want to ask you all a question: Where do you people get this kind of knowledge? Are there any books or lectures that you might recommend? – Galaxy Nov 24 '19 at 07:16
  • 1
    @Galaxy, I'm pretty certain very *few* people get it from dead-tree documents anymore :-) I got my hard-earned knowledge by making a ton of errors then fixing them with research (first with books/magazines, later with bulletin boards and the net). That's a hell of a lot easier nowadays with Stack Overflow :-) – paxdiablo Nov 24 '19 at 07:19
  • This answer is of course correct. But I'm curious as to how we can assign this $$number shell variable to a make variable. I'm attempting to use $$number with the $(word n,text) function, and it seems it can't evaluate $$number or $${number}. You also cannot assign $$number or any variation to a make variable using eval... – krazibiosvn Mar 19 '20 at 23:57
319

If you're using GNU make, you could try

NUMBERS = 1 2 3 4
doit:
        $(foreach var,$(NUMBERS),./a.out $(var);)

which will generate and execute

./a.out 1; ./a.out 2; ./a.out 3; ./a.out 4;
Idelic
  • 14,976
  • 5
  • 35
  • 40
  • 37
    This answer is IMHO better, cause it does not require to use any shell, it's pure makefile (even if it's GNU-specific). – Jocelyn delalande Jan 21 '11 at 16:08
  • 10
    the semicolon is crucial otherwise only the first iteration will execute – Jeremy Leipzig May 18 '12 at 19:55
  • 9
    This solution hides the exit code of `./a.out 1`. `./a.out 2` will be executed regardless. – bobbogo Apr 29 '13 at 10:11
  • 3
    This is not a better answer, because the built in foreach function has limitations on how many elements can be looped over, compared to using the while function in the shell instead. – Alexander Jun 18 '14 at 09:02
  • @bobbogo: sure, because the answer addresses the OP's question. It is quite trivial, as I'm sure you're aware, to make it handle exit codes. – Idelic Jun 22 '14 at 01:27
  • 1
    @Alexander: could you elaborate on what limitation you're referring to? – Idelic Jun 22 '14 at 01:28
  • 1
    @Idelic, yes. For the following makefile and shellscript, the shellscript works (passes the argument to ls), while the Makefile gives an error: **make: execvp: /bin/sh: Argument list too long** Makefile: http://ix.io/d5L Shellscript: http://ix.io/d5M This is a problem in the foreach function that I encountered at work, when an unusually long list of filenames (with long paths as well) were passed to a command. We had to find a workaround. – Alexander Jun 22 '14 at 18:08
  • 2
    @Alexander: As the message says, that's not a limit in GNU make's `foreach`, but in the `exec` system call used by the shell to invoke `ls`. There is no limit, other than memory, for the number of arguments to `foreach`. The script and make file that you point to clearly don't do the same thing, and it is pretty clear why you see the behavior you mention. Again it has nothing to do with limits in the `foreach` function. – Idelic Jun 22 '14 at 23:22
  • @Idelic, even though foreach might in principle handle this, switching away from using foreach solved the problem (even if this was a sideffect). – Alexander Jun 23 '14 at 14:02
  • 2
    Suggest `$(foreach var,$(NUMBERS),./a.out $(var) &&) true` instead so it aborts if any of the commands fail. This fixes the problem that @bobbogo pointed out. – sherbang Jul 02 '19 at 00:45
  • Wouldn't this require the source files to exist 'before' the recipe is executed? Also, wouldn't the expansion also happen even before the recipe is executed? – akhan Nov 25 '19 at 18:47
133

THE major reason to use make IMHO is the -j flag. make -j5 will run 5 shell commands at once. This is good if you have 4 CPUs say, and a good test of any makefile.

Basically, you want make to see something like:

.PHONY: all
all: job1 job2 job3

.PHONY: job1
job1: ; ./a.out 1

.PHONY: job2
job2: ; ./a.out 2

.PHONY: job3
job3: ; ./a.out 3

This is -j friendly (a good sign). Can you spot the boiler-plate? We could write:

.PHONY: all job1 job2 job3
all: job1 job2 job3
job1 job2 job3: job%:
    ./a.out $*

for the same effect (yes, this is the same as the previous formulation as far as make is concerned, just a bit more compact).

A further bit of parameterisation so that you can specify a limit on the command-line (tedious as make does not have any good arithmetic macros, so I'll cheat here and use $(shell ...))

LAST := 1000
NUMBERS := $(shell seq 1 ${LAST})
JOBS := $(addprefix job,${NUMBERS})
.PHONY: all ${JOBS}
all: ${JOBS} ; echo "$@ success"
${JOBS}: job%: ; ./a.out $*

You run this with make -j5 LAST=550, with LAST defaulting to 1000.

bobbogo
  • 14,989
  • 3
  • 48
  • 57
  • 8
    This is definitely the best answer, for the reason that bobbogo mentioned (_-j_). – dbn Oct 04 '12 at 23:28
  • Any particular reason this uses .PHONY suffixes? Are they required for anything? – Seb Dec 11 '12 at 15:14
  • 7
    @seb: Without `.PHONY: all`, make will look for a file called `all`. If such a file exists, make then checks that file's last-changed-time, and the makefile will almost certainly not do what you intended. The `.PHONY` declaration tells make that `all` is a symbolic target. Make will therefore consider the `all` target to always be out of date. Perfect. See the manual. – bobbogo Dec 12 '12 at 15:34
  • Thanks for the clarification bobbogo- might be worth adding something about that to the answer? (although I suppose it's not strictly related to the question) – Seb Jan 03 '13 at 10:06
  • 4
    How does this line work: ${JOBS}: job%: What is the second semicolon for? I didn't see anything in http://www.gnu.org/software/make/manual/make.htm – Joe May 06 '14 at 05:19
  • 2
    @JoeS http://www.gnu.org/software/make/manual/make.html#Static-Pattern (I think you meant _What is the 2nd *colon* for_). _Do not confuse them with the un-nice (IMHO) Pattern Rules_. Static pattern rules are really useful whenever the list of targets can be matched by one of _make_'s noddy patterns (same applies to the dependencies). Here I use one just for the convenience that whatever matched the `%` in the rule is available as `$*` in the recipe. – bobbogo May 06 '14 at 15:04
24

I realize the question is several years old, but this post may still be of use to someone as it demonstrates an approach which differs from the above, and isn't reliant upon either shell operations nor a need for the developer to schpeel out a hardcoded string of numeric values.

the $(eval ....) builtin macro is your friend. Or can be at least.

define ITERATE
$(eval ITERATE_COUNT :=)\
$(if $(filter ${1},0),,\
  $(call ITERATE_DO,${1},${2})\
)
endef

define ITERATE_DO
$(if $(word ${1}, ${ITERATE_COUNT}),,\
  $(eval ITERATE_COUNT+=.)\
  $(info ${2} $(words ${ITERATE_COUNT}))\
  $(call ITERATE_DO,${1},${2})\
)
endef

default:
  $(call ITERATE,5,somecmd)
  $(call ITERATE,0,nocmd)
  $(info $(call ITERATE,8,someothercmd)

That's a simplistic example. It won't scale pretty for large values -- it works, but as the ITERATE_COUNT string will increase by 2 characters (space and dot) for each iteration, as you get up into the thousands, it takes progressively longer to count the words. As written, it doesn't handle nested iteration (you'd need a separate iteration function and counter to do so). This is purely gnu make, no shell requirement (though obviously the OP was looking to run a program each time -- here, I'm merely displaying a message). The if within ITERATE is intended to catch the value 0, because $(word...) will error out otherwise.

Note that the growing string to serve as a counter is employed because the $(words...) builtin can provide an arabic count, but that make does not otherwise support math operations (You cannot assign 1+1 to something and get 2, unless you're invoking something from the shell to accomplish it for you, or using an equally convoluted macro operation). This works great for an INCREMENTAL counter, not so well for a DECREMENT one however.

I don't use this myself, but recently, I had need to write a recursive function to evaluate library dependencies across a multi-binary, multi-library build environment where you need to know to bring in OTHER libraries when you include some library which itself has other dependencies (some of which vary depending on build parameters), and I use an $(eval) and counter method similar to the above (in my case, the counter is used to ensure we don't somehow go into an endless loop, and also as a diagnostic to report how much iteration was necessary).

Something else worth nothing, though not significant to the OP's Q: $(eval...) provides a method to circumvent make's internal abhorrence to circular references, which is all good and fine to enforce when a variable is a macro type (intialized with =), versus an immediate assignment (initialized with :=). There are times you want to be able to use a variable within its own assignment, and $(eval...) will enable you to do that. The important thing to consider here is that at the time you run the eval, the variable gets resolved, and that part which is resolved is no longer treated as a macro. If you know what you're doing and you're trying to use a variable on the RHS of an assignment to itself, this is generally what you want to happen anyway.

  SOMESTRING = foo

  # will error.  Comment out and re-run
  SOMESTRING = pre-${SOMESTRING}

  # works
  $(eval SOMESTRING = pre${SOMESTRING}

default:
  @echo ${SOMESTRING}

Happy make'ing.

s.straw
  • 241
  • 2
  • 2
22

For cross-platform support, make the command separator (for executing multiple commands on the same line) configurable.

If you're using MinGW on a Windows platform for example, the command separator is &:

NUMBERS = 1 2 3 4
CMDSEP = &
doit:
    $(foreach number,$(NUMBERS),./a.out $(number) $(CMDSEP))

This executes the concatenated commands in one line:

./a.out 1 & ./a.out 2 & ./a.out 3 & ./a.out 4 &

As mentioned elsewhere, on a *nix platform use CMDSEP = ;.

cherno
  • 808
  • 9
  • 15
9

Maybe you can use:

xxx:
    for i in `seq 1 4`; do ./a.out $$i; done;
Donwellus
  • 99
  • 1
  • 1
7

This is not really a pure answer to the question, but an intelligent way to work around such problems:

instead of writing a complex file, simply delegate control to for instance a bash script like: makefile

foo : bar.cpp baz.h
    bash script.sh

and script.sh looks like:

for number in 1 2 3 4
do
    ./a.out $number
done
Willem Van Onsem
  • 443,496
  • 30
  • 428
  • 555
  • I have got stuck at a step while writing a Makefile. I have following code : set_var: @ NUM=0 ; while [[ $$NUM < 1 ]]; do \ echo "I am here"; \ echo $$NUM dump$${NUM}.txt; \ var="SSA_CORE$${NUM}_MAINEXEC" ; \ echo $$var ; \ var1=`eval echo \$${$(var)}`; \ echo $$var1; \ (( NUM = NUM + 1)); \ done all:set_var here SSA_CORE0_MAINEXEC is an environment variable which is already set.So I want that value to get evaluated or printed using variable var1. I tried it as shown above but not working. pLease help. – XYZ_Linux Jan 17 '14 at 08:38
  • 2
    This is indeed an easy workaround, however it keeps you from using the nice "make -j 4" option to have processes run in parallel. – TabeaKischka Jul 19 '18 at 20:35
  • 1
    @TabeaKischka: indeed. It was not the "*recommended*" way, but it is more meant if one needs some features that are not offered by a Makefile, then one can fallback to an implementation in `bash` and thus use the features. The loop is one of the features for which this can be demonstrated. – Willem Van Onsem Jul 19 '18 at 20:48
3

You can use set -e as a prefix for the for-loop. Example:

all:
    set -e; for a in 1 2 3; do /bin/false; echo $$a; done

make will exit immediately with an exit code <> 0.

1

Although the GNUmake table toolkit has a true while loop (whatever that means in GNUmake programming with its two or three phases of execution), if the thing which is needed is an iterative list, there is a simple solution with interval. For the fun of it, we convert the numbers to hex too:

include gmtt/gmtt.mk

# generate a list of 20 numbers, starting at 3 with an increment of 5
NUMBER_LIST := $(call interval,3,20,5)

# convert the numbers in hexadecimal (0x0 as first operand forces arithmetic result to hex) and strip '0x'
NUMBER_LIST_IN_HEX := $(foreach n,$(NUMBER_LIST),$(call lstrip,$(call add,0x0,$(n)),0x))

# finally create the filenames with a simple patsubst
FILE_LIST := $(patsubst %,./a%.out,$(NUMBER_LIST_IN_HEX))

$(info $(FILE_LIST))

Output:

./a3.out ./a8.out ./ad.out ./a12.out ./a17.out ./a1c.out ./a21.out ./a26.out ./a2b.out ./a30.out ./a35.out ./a3a.out ./a3f.out ./a44.out ./a49.out ./a4e.out ./a53.out ./a58.out ./a5d.out ./a62.out
Vroomfondel
  • 2,704
  • 1
  • 15
  • 29
1

A simple, shell/platform-independent, pure macro solution is ...

# GNU make (`gmake`) compatible; ref: <https://www.gnu.org/software/make/manual>
define EOL

$()
endef
%sequence = $(if $(word ${1},${2}),$(wordlist 1,${1},${2}),$(call %sequence,${1},${2} $(words _ ${2})))

.PHONY: target
target:
    $(foreach i,$(call %sequence,10),./a.out ${i}${EOL})
rivy
  • 1,570
  • 1
  • 15
  • 30
1

Dynamically assign variables in the loop

The promblem with the for number in 1 2 3 4 ; do \ ...-solution is, that no variables can be assigned within the loop. $(eval VAR=...) can only be used, if the content of the assignment is known at the beginning of the target execution. If the assignment depends on the loop-variable, VAR will be empty.

To circumvent this issue, one can use the target functionality to model a loop. The following example takes the n-th file from SRC / OBJ and processes them together. Using this construction you can even use $(eval ...) to process the loop variable, as demonstrated with VAR3.

makefile

SRC = f1.c f2.cpp f3.cpp
OBJ = f1.o f2.o f3.o

SRC2 = $(addsuffix _,$(SRC))
JOIN = $(join $(SRC2),$(OBJ))

PHONY: all
all : info loop

loop : $(JOIN)

$(JOIN) :
    @# LOOP - CONTENT
    @echo "TARGET: $@"
    $(eval VAR1=$(word 1,$(subst _, ,$@)))
    @echo "VAR1: "$(VAR1)
    $(eval VAR2=$(word 2,$(subst _, ,$@)))
    @echo "VAR2: "$(VAR2)
    $(eval VAR3=$(subst .o,.x,$(VAR2)))
    @echo "You can even substitute you loop variable VAR3: "$(VAR3)
    #g++ -o $(VAR2) $(VAR1)
    @echo

PHONY: info
info:
    @printf "\n"
    @echo "JOIN: "$(JOIN)
    @printf "\n"

output

$ make

JOIN: f1.c_f1.o f2.cpp_f2.o f3.cpp_f3.o

TARGET: f1.c_f1.o
VAR1: f1.c
VAR2: f1.o
You can even substitute you loop variable VAR3: f1.x
#g++ -o f1.o f1.c

TARGET: f2.cpp_f2.o
VAR1: f2.cpp
VAR2: f2.o
You can even substitute you loop variable VAR3: f2.x
#g++ -o f2.o f2.cpp

TARGET: f3.cpp_f3.o
VAR1: f3.cpp
VAR2: f3.o
You can even substitute you loop variable VAR3: f3.x
#g++ -o f3.o f3.cpp
Markus Dutschke
  • 9,341
  • 4
  • 63
  • 58
1

Here is an example with dynamic range

default: all


.DEFAULT:
    for i in $(shell seq 1 $(RUN_COUNT)) ; do \
        echo "RUN_COUNT $$i:" || exit 1; \
    done

Use || exit 1 to exit when ctrl + c is pressed. It will also make sure that a single loop execution failure will exit. (In this trivial example, this is not useful. But if you run a command that can fail, this is helpful).

Invoke using RUN_COUNT=10 make

Sample output:

RUN_COUNT 1:
RUN_COUNT 2:
RUN_COUNT 3:
RUN_COUNT 4:
RUN_COUNT 5:
RUN_COUNT 6:
RUN_COUNT 7:
...
srgsanky
  • 671
  • 1
  • 11
  • 16
0

This answer, just as that of @Vroomfondel aims to circumvent the loop problem in an elegant way.

My take is to let make generate the loop itself as an imported makefile like this:

include Loop.mk
Loop.mk:Loop.sh
     Loop.sh > $@

The shell script can the be as advanced as you like but a minimal working example could be

#!/bin/bash
LoopTargets=""
NoTargest=5
for Target in `seq $NoTargest` ; do
    File="target_${Target}.dat"
    echo $File:data_script.sh
    echo $'\t'./data_script.ss $Target
    LoopTargets="$LoopTargets $File"
done
echo;echo;echo LoopTargets:=$LoopTargets

which generates the file

target_1.dat:data_script.sh
    ./data_script.ss 1
target_2.dat:data_script.sh
    ./data_script.ss 2
target_3.dat:data_script.sh
    ./data_script.ss 3
target_4.dat:data_script.sh
    ./data_script.ss 4
target_5.dat:data_script.sh
    ./data_script.ss 5


LoopTargets:= target_1.dat target_2.dat target_3.dat target_4.dat target_5.dat

And advantage there is that make can itself keep track of which files have been generated and which ones need to be (re)generated. As such, this also enables make to use the -j flag for parallelization.

Mikael Fremling
  • 720
  • 2
  • 8
  • 24
0

This worked for me:

NUM=4

a-out:
    for (( i=1; i<=${NUM}; i++ )) \
    do \
        ./a.out $$i ; \
    done
-1
#I have a bunch of files that follow the naming convention
#soxfile1  soxfile1.o  soxfile1.sh   soxfile1.ini soxfile1.txt soxfile1.err
#soxfile2  soxfile2.o   soxfile2.sh  soxfile2.ini soxfile2.txt soxfile2.err
#sox...        ....        .....         ....         ....        ....
#in the makefile, only select the soxfile1.. soxfile2... to install dir
#My GNU makefile solution follows:
tgt=/usr/local/bin/          #need to use sudo
tgt2=/backup/myapplication/  #regular backup 

install:
        for var in $$(ls -f sox* | grep -v '\.' ) ; \
        do \
                sudo  cp -f $$var ${TGT} ;     \
                      cp -f  $$var ${TGT2} ;  \
        done


#The ls command selects all the soxfile* including the *.something
#The grep command rejects names with a dot in it, leaving  
#My desired executable files in a list. 
-1

Version

Code

files := $(wildcard ./*.txt ./**/*.go */**/*.js )

showFileFunc = echo "$(abspath ${1})\${2}"
delFileFunc = del "$(abspath ${1})\${2}"
cmdSplit = &
targetDisplay:
    $(foreach curFile, ${files}, ${call showFileFunc,${dir $(curFile)},${notdir $(curFile)}} ${cmdSplit})
targetDelete:
    $(foreach curFile, ${files}, ${call delFileFunc,${dir $(curFile)},${notdir $(curFile)}} ${cmdSplit})

Test Directory

Makefile
1.txt
 myDir
   - foo.go
   - bar.go
   - subDir
     - qoo.js

Test CMD & output

make showFile -s

output:
"C:/...\1.txt"
"C:/.../myDir\bar.go"
"C:/.../myDir\foo.go"          // since `.//**.js`
"C:/.../myDir/subDir\qoo.js"   // `.//**.js` can't but `*/**/*.js` is OK

Reference

Carson
  • 6,105
  • 2
  • 37
  • 45