How to source a script in a Makefile?

Question

Is there a better way to source a script, which sets env vars, from within a makefile?

FLAG ?= 0
ifeq ($(FLAG),0)
export FLAG=1
/bin/myshell -c '<source scripts here> ; $(MAKE) $@'
else
...targets...
endif

Sounds like what you really want is to write a script that sources your script to set envvars and then runs make, rather than having make source the script itself... — Chris Dodd, Sep 21 '11 at 23:39
I saw a great answer here: https://unix.stackexchange.com/a/235254/18152 and here: https://blog.153.io/2016/04/18/source-a-shell-script-in-make/ — zx1986, Dec 24 '18 at 07:00

score 157 · Answer 1 · answered Apr 23 '17 at 00:35

157

Makefile default shell is /bin/sh which does not implement source.

Changing shell to /bin/bash makes it possible:

# Makefile

SHELL := /bin/bash

rule:
    source env.sh && YourCommand

answered Apr 23 '17 at 00:35

PureW

4,568
3
19
27

9

@VivekAditya This is, of course, a vastly useful thing. But notice that the OP was asking to set variable for use by `make` and not just for use in action lines. This is how to do the latter, but there is no direct way to do the former. – dmckee --- ex-moderator kitten Apr 09 '19 at 22:59
1

You can easily set `SHELL` to your own wrapper to accomplish this. Shown here https://gist.github.com/ingydotnet/99f7e259319f6b90e50a754f3053be7f – ingydotnet Sep 24 '19 at 18:48
5

This didn't seem to work for me? – ngood97 Jan 22 '22 at 01:25
8

This does not work on macOS – A. Gille Jun 03 '22 at 15:42
And for /bin/sh just use `. env.sh` to accomplish this. – MKesper Jun 21 '23 at 13:07

score 64 · Accepted Answer · edited Aug 31 '15 at 14:43

64

To answer the question as asked: you can't.

The basic issue is that a child process can not alter the parent's environment. The shell gets around this by not forking a new process when source'ing, but just running those commands in the current incarnation of the shell. That works fine, but make is not /bin/sh (or whatever shell your script is for) and does not understand that language (aside from the bits they have in common).

Chris Dodd and Foo Bah have addressed one possible workaround, so I'll suggest another (assuming you are running GNU make): post-process the shell script into make compatible text and include the result:

shell-variable-setter.make: shell-varaible-setter.sh
    postprocess.py @^

# ...
else
include shell-variable-setter.make
endif

messy details left as an exercise.

edited Aug 31 '15 at 14:43

gsamaras

71,951
46
188
305

answered Sep 22 '11 at 00:35

dmckee --- ex-moderator kitten

98,632
24
142
234

4

You "can't" source a script for a whole Makefile, but you *can* do that for a single recipe. – Yaroslav Nikitenko Aug 16 '18 at 13:55
Yes `source compilationEnv.sh; $(CC) -c -o $< $^` or similar will work as action line. – dmckee --- ex-moderator kitten Apr 04 '19 at 15:04
3

You can use `include .env` (or any key-value file) to make variables accessible to the whole Makefile and then pass only the needed ones into the respective recipe. – rypel Aug 24 '19 at 06:38

Foo Bah · Answer 3 · 2011-09-21T23:45:36.847

26

If your goal is to merely set environment variables for Make, why not keep it in Makefile syntax and use the include command?

include other_makefile

If you have to invoke the shell script, capture the result in a shell command:

JUST_DO_IT=$(shell source_script)

the shell command should run before the targets. However this won't set the environment variables.

If you want to set environment variables in the build, write a separate shell script that sources your environment variables and calls make. Then, in the makefile, have the targets call the new shell script.

For example, if your original makefile has target a, then you want to do something like this:

# mysetenv.sh
#!/bin/bash
. <script to source>
export FLAG=1
make "$@" 

# Makefile
ifeq($(FLAG),0)
export FLAG=1
a: 
    ./mysetenv.sh a
else
a:
    .. do it
endif

edited Sep 21 '11 at 23:45

answered Sep 21 '11 at 23:13

Foo Bah

25,660
5
55
79

7

I have no control over the script. Must source it. – brooksbp Sep 21 '11 at 23:22
Will this capture the environment variables if they aren't exported? From first glance it seems like that invokes a new shell, runs command, then exits and returns to old environment..? – brooksbp Sep 21 '11 at 23:35
@brooksbp The shell command will not capture the environment variables. But, if you write a shell script that sources the original one, then it will. I will explain it further – Foo Bah Sep 21 '11 at 23:40

score 21 · Answer 4 · answered Dec 31 '14 at 00:39

21

Using GNU Make 3.81 I can source a shell script from make using:

rule:
<tab>source source_script.sh && build_files.sh

build_files.sh "gets" the environment variables exported by source_script.sh.

Note that using:

rule:
<tab>source source_script.sh
<tab>build_files.sh

will not work. Each line is ran in its own subshell.

answered Dec 31 '14 at 00:39

Samuel

8,063
8
45
41

1

Note that keeping `source` and commands on the same line becomes necessary with GNU Make 4.1 (and perhaps 4.0 too). Using 3.81, I can use different lines. For portability, one line works across these versions. Invoking an external script is more readable if the line gets too long, though! – Eric Platon Feb 12 '18 at 05:17
Are you sure? Multiple lines didn't work for me in 3.81, as I mentioned in the answer. Make runs each line in its own subshell, so multiple lines shouldn't work on any version. Maybe you have an uncommon OS or make build? – Samuel Feb 13 '18 at 17:28
2

multiple lines works as long as you end each line with `\\` – user5359531 Oct 10 '18 at 02:28
@user5359531 Yep, that's true. – Samuel Oct 10 '18 at 14:48
Use special target .ONESHELL: to get rid of those \ es. – MKesper Jun 21 '23 at 13:10

score 12 · Answer 5 · answered May 10 '13 at 21:09

12

This works for me. Substitute env.sh with the name of the file you want to source. It works by sourcing the file in bash and outputting the modified environment, after formatting it, to a file called makeenv which is then sourced by the makefile.

IGNORE := $(shell bash -c "source env.sh; env | sed 's/=/:=/' | sed 's/^/export /' > makeenv")                         
include makeenv

answered May 10 '13 at 21:09

gdw2

7,558
4
46
49

1

`ifdef _intel64onosx`
`GCC=/opt/intel/bin/icc`
`CFLAGS= -m64 -g -std=c99` `IGNORE :=$(shell bash -c "source /opt/intel/bin/iccvars.sh intel64; env | sed 's/=/:=/' | sed 's/^/export /' > makeenv")` `include makeenv` – revher Dec 22 '14 at 22:46
Nice for setting correct compiler environments (here 64-bit). Thank you. – revher Dec 22 '14 at 22:54
Is the syntax to execute this: "make IGNORE"? – John Oct 01 '19 at 07:15
@John, no. Just `make`. – gdw2 Oct 01 '19 at 17:28
What if the values of the variables in your `env.sh` contain special Make characters such as `#` or `$`. When you do the `include makeenv` those values will get all messed up. – donhector Feb 07 '20 at 01:16

artless noise · Answer 6 · 2012-12-06T16:15:37.413

8

Some constructs are the same in the shell and in GNU Make.

var=1234
text="Some text"

You can alter your shell script to source the defines. They must all be simple name=value types.

Ie,

[script.sh]

. ./vars.sh

[Makefile]

include vars.sh

Then the shell script and the Makefile can share the same 'source' of information. I found this question because I was looking for a manifest of common syntax that can be used in Gnu Make and shell scripts (I don't care which shell).

Edit: Shells and make understand ${var}. This means you can concatenate, etc, var="One string" var=${var} "Second string"

edited Dec 06 '12 at 16:15

answered Dec 05 '12 at 20:16

artless noise

21,212
6
68
105

Err, a shell script like the above creates no **messy details** as in the accepted answer. It ends up being a lot like a [tag:kbuild] file. – artless noise Jan 27 '15 at 22:58
1

`shell` variables and `make` variables are not always transparently interchangeable. When you `include` them in your `Makefile`, they must comply with `make` syntax, which imposes certain characters to be escaped. For example a shell variable value like `SECRET='12#34$56'` in your `.vars.sh` will cause you trouble when go use it after `include vars.sh` – donhector Feb 07 '20 at 01:36
1

I think there are two use cases; the OP has no control over the script. So your point is very good as you are most likely to get data that needs to be escape. These are "%$#". If the use case is to share variables between tools which use environment variables and make, then this solution can work. It has an advantage of only one source of information and no extra processes. If you don't have control over the shell script, this answer is not as useful. Also questionable as why you would want "$%#" in a variable name. If you compute items, you need to follow the 'Edit:" portion. – artless noise Feb 08 '20 at 14:49

score 3 · Answer 7 · answered Mar 10 '16 at 18:45

I really like Foo Bah's answer where make calls the script, and the script calls back to make. To expand on that answer I did this:

# Makefile
.DEFAULT_GOAL := all 

ifndef SOME_DIR

%:
<tab>. ./setenv.sh $(MAKE) $@

else

all:
<tab>...

clean:
<tab>...

endif

--

# setenv.sh
export SOME_DIR=$PWD/path/to/some/dir

if [ -n "$1" ]; then
    # The first argument is set, call back into make.
    $1 $2
fi

This has the added advantage of using $(MAKE) in case anyone is using a unique make program, and will also handle any rule specified on the command line, without having to duplicate the name of each rule in the case when SOME_DIR is not defined.

If you have an older version of make and .DEFAULT_GOAL isn't supported then the first branch of the ifndef will fail with "make: *** No targets. Stop.". You can fix this by adding "all" to the implicit rule, like this: "all %:". Note that this will cause a " *** mixed implicit and normal rules: deprecated syntax" warning with newer versions of make. — Samuel, Mar 25 '16 at 20:50

Paul A Jungwirth · Answer 8 · 2017-03-01T15:55:32.723

If you want to get the variables into the environment, so that they are passed to child processes, then you can use bash's set -a and set +a. The former means, "When I set a variable, set the corresponding environment variable too." So this works for me:

check:
    bash -c "set -a && source .env.test && set +a && cargo test"

That will pass everything in .env.test on to cargo test as environment variables.

Note that this will let you pass an environment on to sub-commands, but it won't let you set Makefile variables (which are different things anyway). If you need the latter, you should try one of the other suggestions here.

score 2 · Answer 9 · answered Feb 13 '16 at 23:50

My solution to this: (assuming you're have bash, the syntax for $@ is different for tcsh for instance)

Have a script sourceThenExec.sh, as such:

#!/bin/bash
source whatever.sh
$@

Then, in your makefile, preface your targets with bash sourceThenExec.sh, for instance:

ExampleTarget:
    bash sourceThenExec.sh gcc ExampleTarget.C

You can of course put something like STE=bash sourceThenExec.sh at the top of your makefile and shorten this:

ExampleTarget:
    $(STE) gcc ExampleTarget.C

All of this works because sourceThenExec.sh opens a subshell, but then the commands are run in the same subshell.

The downside of this method is that the file gets sourced for each target, which may be undesirable.

tankthinks · Answer 10 · 2018-08-21T13:39:47.440

1

Depending on your version of Make and enclosing shell, you can implement a nice solution via eval, cat, and chaining calls with &&:

ENVFILE=envfile

source-via-eval:
  @echo "FOO: $${FOO}"
  @echo "FOO=AMAZING!" > $(ENVFILE)
  @eval `cat $(ENVFILE)` && echo "FOO: $${FOO}"

And a quick test:

> make source-via-eval
FOO:
FOO: AMAZING!

edited Aug 21 '18 at 13:39

answered Aug 21 '18 at 13:28

tankthinks

973
6
9

This works for me on Ubuntu 20.4. Thanks! – jrosell Feb 10 '22 at 09:07

score 1 · Answer 11 · answered Dec 02 '20 at 22:33

1

An elegant solution found here:

ifneq (,$(wildcard ./.env))
    include .env
    export
endif

answered Dec 02 '20 at 22:33

Sanpi

420
7
18

score 0 · Answer 12 · answered Jan 07 '16 at 09:50

If you need only a few known variables exporting in makefile can be an option, here is an example of what I am using.

$ grep ID /etc/os-release 

ID=ubuntu
ID_LIKE=debian


$ cat Makefile

default: help rule/setup/lsb

source?=.

help:
        -${MAKE} --version | head -n1

rule/setup/%:
        echo ID=${@F}

rule/setup/lsb: /etc/os-release
        ${source} $< && export ID && ${MAKE} rule/setup/$${ID}


$ make

make --version | head -n1
GNU Make 3.81
. /etc/os-release && export ID && make rule/setup/${ID}
make[1]: Entering directory `/tmp'
echo ID=ubuntu
ID=ubuntu

-- http://rzr.online.fr/q/gnumake

score 0 · Answer 13 · answered Jun 24 '22 at 09:35

Assuming GNU make, can be done using a submake. Assuming that the shell script that exports the variables is include.sh in the current directory, move your Makefile to realmake.mk. Create a new Makefile:

all:
    @. ./include.sh; \
    $(MAKE) -f realmake.mk $(MAKECMDGOALS)

$(MAKECMDGOALS):
    +@. ./include.sh; \
    $(MAKE) -f realmake.mk $(MAKECMDGOALS)

Pay attention to the ./ preceding include.sh.

score -2 · Answer 14 · answered Aug 19 '16 at 13:12

-2

Another possible way would be to create a sh script, for example run.sh, source the required scripts and call make inside the script.

#!/bin/sh
source script1
source script2 and so on

make

answered Aug 19 '16 at 13:12

Veera

861
1
9
18

score -2 · Answer 15 · answered Sep 01 '17 at 14:41

-2

target: output_source
    bash ShellScript_name.sh

try this it will work, the script is inside the current directory.

answered Sep 01 '17 at 14:41

amit pandya

1,384
13
22

How to source a script in a Makefile?

15 Answers15

Linked