How to write commands with multiple lines in Dockerfile while preserving the new lines?

Question

I want to write the following RUN command in the Dockerfile. But, docker is not preserving the new lines.

RUN echo "[repo] \
name            = YUM Repository \
baseurl         = https://example.com/packages/ \
enabled         = 1 \
gpgcheck        = 0" > /etc/yum.repos.d/Repo.repoxyz

I know that \ at the end of each line escapes the new line. But, is there any way that I can write multiple lines preserving the new line?

Are you using non *nix platform? Because this works fine for me on Linux. — user, Feb 19 '16 at 20:50

score 175 · Accepted Answer · edited May 05 '20 at 08:11

175

You can use what is called "ANSI-C quoting" with $'...'. It was originally a ksh93 feature but it is now available in bash, zsh, mksh, FreeBSD sh and in busybox's ash (but only when it is compiled with ENABLE_ASH_BASH_COMPAT).

As RUN uses /bin/sh as shell by default you are required to switch to something like bash first by using the SHELL instruction.

Start your command with $', end it with ' and use \n\ for newlines, like this:

SHELL ["/bin/bash", "-c"]

RUN echo $'[repo] \n\
name            = YUM Repository \n\
baseurl         = https://example.com/packages/ \n\
enabled         = 1 \n\
gpgcheck        = 0' > /etc/yum.repos.d/Repo.repoxyz

edited May 05 '20 at 08:11

Forage

2,726
2
18
20

answered Oct 30 '15 at 15:33

Daniel Zolnai

16,487
7
59
71

Is the $ syntax after echo a feature of Dockerfile? Because, I couldn't find about it in the documentation. – Venkata Jaswanth Nov 02 '15 at 17:29
4

It is bash syntax. See this question for more info: http://stackoverflow.com/a/11966402/1395437 – Daniel Zolnai Nov 02 '15 at 18:25
18

Please, explain what you're doing, don't just drop an opaque solution – Édouard Lopez Sep 22 '16 at 09:34
What about `EXPOSE`, or some other command that's not `RUN`? Can you use the `\` notation? I can't find anything only about that because all the examples are with `RUN`. – Apr 06 '18 at 07:25
35

A word of caution: the `$' ... \n\ ` technique depends on the shell that docker `RUN` uses being `bash`. On some systems (such as Ubuntu) the shell RUN uses is `/bin/sh` which is often a link to `dash` which is NOT `bash` and does not understand the `$'` syntax. – Anon Aug 04 '18 at 07:30
3

as @Anon said, this will NOT WORK if run anywhere else than bash. (doesn't work in `Oh My Zsh` either) – Rafa Apr 24 '19 at 10:03
3

@Anon well it does work in Alpine linux (currently I am on 3.10.2) which uses shell. – suchoss Oct 15 '19 at 07:35
1

@suchoss I stand corrected re `$'...'` only working in `bash` (Alpine uses busybox's `ash` as its default shell). The `$'...'` feature is known as "ANSI-C quoting" but it's not a POSIX shell feature. According to https://unix.stackexchange.com/a/371873/109111 it was originally a `ksh93` feature but it is now available in `bash`, `zsh`, `mksh`, FreeBSD `sh` and in busybox's `ash` (but only when it is compiled with `ENABLE_ASH_BASH_COMPAT`). It is NOT available in `dash` which is Ubuntu's default `/bin/sh`. – Anon Oct 21 '19 at 09:49
2

It doesn't work with variables. – Theodore R. Smith Dec 27 '21 at 22:10
If I need to use an environment variable in the script and pass the value at run time. How to do it? – panc Dec 29 '21 at 19:23
1

I think `RUN` command can be executed as `RUN /bin/bash -c $'echo ...` , without using `SHELL` which affects the shell for subsequent docker commands as well. – murtiko Mar 04 '22 at 12:50

score 92 · Answer 2 · edited Aug 12 '22 at 05:59

92

Use printf to allow a single RUN command to output multiple lines of text, using \n to insert newlines.

Executing:

RUN printf 'example\ntext\nhere' >> example.txt

appends:

example
text
here

to the file example.txt

edited Aug 12 '22 at 05:59

Tracy Logan

84
1
4

answered Apr 19 '16 at 14:12

Cristian Todea

1,347
11
23

This worked for me! Echo had newline characters in the file as "\n" instead of an actual new line. – Deepthi Feb 02 '23 at 09:57

Paulo Fidalgo · Answer 3 · 2019-05-16T15:51:05.947

39

You can use:

RUN echo -e "\
[repo] \n\
name            = YUM Repository \n\
baseurl         = https://example.com/packages/ \n\
enabled         = 1 \n\
gpgcheck        = 0\
" > /etc/yum.repos.d/Repo.repoxyz

This way you will have a quick way to check what the file contents are. You just need to be aware that you need to end every line with \ and insert the \n when needed.

edited May 16 '19 at 15:51

answered Jan 07 '19 at 12:43

Paulo Fidalgo

21,709
7
99
115

3

This requires `echo -e` to interpret `\n` – Patrick Bergner May 14 '19 at 16:58
7

Using the `-e` option gave me an unexpected behaviour. The option is not interpreted as such but as part of the text to be printed. – Martin Tovmassian Nov 03 '21 at 08:18
Good solution. Interestingly, if the first line after `RUN echo -e ...` is *#!/bin/bash \n\* it doesn't end up in the file. Fixed it by doing `echo -e '#!/bin/bash \n\` instead. And I'm using `'` instead of `"` to avoid parameter expansion. Apologies for the formatting, not working as expected! – Nagev Oct 18 '22 at 16:16

Sergey · Answer 4 · 2023-06-29T07:44:26.820

19

I ended up using a combination of the examples listed above since the new line \n did not work with echo.

RUN printf 'example \n\
text \n\
here' >> example.txt

It produces the following, as expected:

example
text
here

edited Jun 29 '23 at 07:44

answered Dec 09 '19 at 09:09

Sergey

3,253
2
33
55

2

How is your answer differs from CTodea's one? – The Godfather Jul 14 '20 at 17:37
2

@TheGodfather That's a multiline example. Better answers the original question too. – Sergey Jul 19 '20 at 08:29

score 14 · Answer 5 · edited Feb 03 '23 at 20:24

14

As of Docker 18.09 and Dockerfile syntax 1.4, Dockerfiles support heredocs (ie what you're looking for here) natively!

Enable BuildKit, eg by setting DOCKER_BUILDKIT=1.
Add this line to the top of your Dockerfile:
```
# syntax=docker/dockerfile:1.4
```
or as recommended in the documentation
```
# syntax=docker/dockerfile:1
```

Rewrite your heredoc like so:

COPY <<EOF /etc/yum.repos.d/Repo.repoxyz
[repo]
name            = YUM Repository
baseurl         = https://example.com/packages/
enabled         = 1
gpgcheck        = 0
EOF

You can also use this to run multiple bash commands in a single RUN block, etc. More details: Docker blog post, Dockerfile syntax docs.

edited Feb 03 '23 at 20:24

Igor

349
1
6

answered Feb 20 '22 at 19:43

ryan

2,687
1
29
38

1

In the blog of Docker, the line is `# syntax=docker/dockerfile:1.3-labs`. – Shuai Mar 09 '22 at 11:43
What is your example doing? Is it writing to a file named `/etc/yum.repos.d/Repo.repoxyz` ? I tried that syntax and it does not create a file – red888 Apr 05 '22 at 17:09
@red888 hmm! Yes, that's what it's doing, same as OP's. Not sure why these steps aren't working for you, they are for me. – ryan Apr 06 '22 at 20:58

score 4 · Answer 6 · answered May 03 '19 at 14:31

May be it's help you ( https://github.com/jen-soft/pydocker )

[ Dockerfile.py ]

from pydocker import DockerFile  # sudo pip install -U pydocker

d = DockerFile(base_img='debian:8.2', name='jen-soft/custom-debian:8.2')

d.RUN_bash_script('/opt/set_repo.sh', r'''
cat >/etc/apt/sources.list <<EOL
deb     http://security.debian.org/ jessie/updates main
deb-src http://security.debian.org/ jessie/updates main
EOL
apt-get clean && apt-get update
''')

d.EXPOSE = 80
d.WORKDIR = '/opt'
d.CMD = ["python", "--version"]

# d.generate_files()
d.build_img()

# sudo wget -qO- https://get.docker.com/ | sh

python Dockerfile.py
docker images

score -5 · Answer 7 · answered Aug 07 '20 at 07:28

You can execute RUN several times to complete your file:

RUN echo "[repo]" >> /etc/yum.repos.d/Repo.repoxyz
RUN echo "name            = YUM Repository" >> /etc/yum.repos.d/Repo.repoxyz
RUN echo "baseurl         = https://example.com/packages/" >> /etc/yum.repos.d/Repo.repoxyz
RUN echo "enabled         = 1" >> /etc/yum.repos.d/Repo.repoxyz
RUN echo "gpgcheck        = 0" >> /etc/yum.repos.d/Repo.repoxyz

This may not be the optimal solution because it creates a new layer for every RUN command. Still, every layer will be as big as the change you make, which in this case it's in the order of Bytes (first RUN layer should be 7-byte).

The benefit of this solution is that it will work with all shells.

Probably better to concat these commands with `&&` for better caching purposes, reduced logging and faster `Dockerfile` build times — JohannesB, Sep 12 '20 at 08:47
This solution adds too many layer to your image which will increase your build time. — Erfun, Mar 24 '21 at 13:28

How to write commands with multiple lines in Dockerfile while preserving the new lines?

7 Answers7

Linked