138

If the following example, which sets the IFS environment variable to a line feed character...

IFS=$'\n'
  • What does the dollar sign mean exactly?
  • What does it do in this specific case?
  • Where can I read more on this specific usage (Google doesn't allow special characters in searches and I don't know what to look for otherwise)?

I know what the IFS environment variable is, and what the \n character is (line feed), but why not just use the following form: IFS="\n" (which does not work)?

For example, if I want to loop through every line of a file and want to use a for loop, I could do this:

for line in (< /path/to/file); do
    echo "Line: $line"
done

However, this won't work right unless IFS is set to a line feed character. To get it to work, I'd have to do this:

OLDIFS=$IFS
IFS=$'\n'
for line in (< /path/to/file); do
    echo "Line: $line"
done
IFS=$OLDIFS

Note: I don't need another way for doing the same thing, I know many other already... I'm only curious about that $'\n' and wondered if anyone could give me an explanation on it.

codeforester
  • 39,467
  • 16
  • 112
  • 140
Yanick Girouard
  • 1,383
  • 2
  • 9
  • 4

7 Answers7

174

Normally bash doesn't interpret escape sequences in string literals. So if you write \n or "\n" or '\n', that's not a linebreak - it's the letter n (in the first case) or a backslash followed by the letter n (in the other two cases).

$'somestring' is a syntax for string literals with escape sequences. So unlike '\n', $'\n' actually is a linebreak.

mikemaccana
  • 110,530
  • 99
  • 389
  • 494
sepp2k
  • 363,768
  • 54
  • 674
  • 675
  • 2
    Not exactly so -- `\n` is just an (escaped) letter n. You are right that `'\n'` and `"\n"` are backlash followed by n. – Roman Cheplyaka Nov 08 '10 at 23:08
  • 17
    Note that `$'\n'` is bash specific -- it won't work in a POSIX shell (`/bin/sh`). To get the same effect in a POSIX-compliant manner, you can type `IFS='`, then hit return to type an actual newline character, then type the closing `'` – Richard Hansen Jun 21 '11 at 16:52
  • 25
    `IFS=$(echo -e '\n')` should also do it in a POSIX-compatible way. – Vineet Oct 06 '11 at 15:54
  • 14
    @Vineet - it gave me pause to dispute an upvoted comment. While this *is* Posix-correct, it doesn't work - The command substitution operators in bash remove all trailing newline characters. See [this for more detail](http://unix.stackexchange.com/questions/17732/where-has-the-trailing-newline-char-gone-from-my-command-substitution). – Digital Trauma Oct 05 '13 at 02:38
  • Make sure that the script starts with "!/bin/bash" and not "!/bin/sh". Not relevant but may be a pitfall. – nvd Sep 04 '14 at 18:51
  • 9
    @DigitalTrauma I think it's not even POSIX: `-e` is not defined, and `\n` without `-e` works as an XSI extension: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html#tag_20_37 . `printf '\n'` rocks ;) – Ciro Santilli OurBigBook.com Oct 10 '14 at 10:23
  • A rather unexpected side effect of using `IFS='\n'` (without the dollar sign) is that your user name gets truncated if it has an `n` in it (This bug is how I got to this thread). `echo $USER` yields `brade` instead of `braden` in my case. – Braden Best Dec 27 '16 at 19:06
  • @Richard Hansen: It is not working in my bash. Has this changed in newer versions of bash? I am using 4.2.46(2)-release – anishjp Nov 28 '19 at 13:59
  • 1
    Here is an actual POSIX solution: `nl="$(printf '\nx')"; nl="${nl%x}"; IFS="$nl"`. By printing an extra character with our newline, the newline isn't lost under command substitution. Then we just use parameter expansion to remove the extra character. – Andrey Kaipov May 28 '20 at 19:01
27

Just to give the construct its official name: strings of the form $'...' are called ANSI C-quoted strings.

That is, as in [ANSI] C strings, backlash escape sequences are recognized and expanded to their literal equivalent (see below for the complete list of supported escape sequences).

After this expansion, $'...' strings behave the same way as '...' strings - i.e., they're treated as literals NOT subject to any [further] shell expansions.

For instance, $'\n' expands to a literal newline character - which is something a regular bash string literal (whether '...' or "...") cannot do.[1]

Another interesting feature is that ANSI C-quoted strings can escape ' (single quotes) as \', which, '...' (regular single-quoted strings) cannot:

echo $'Honey, I\'m home' # OK; this cannot be done with '...'

List of supported escape sequences:

Backslash escape sequences, if present, are decoded as follows:

\a alert (bell)

\b backspace

\e \E an escape character (not ANSI C)

\f form feed

\n newline

\r carriage return

\t horizontal tab

\v vertical tab

\ backslash

\' single quote

\" double quote

\nnn the eight-bit character whose value is the octal value nnn (one to three digits)

\xHH the eight-bit character whose value is the hexadecimal value HH (one or two hex digits)

\uHHHH the Unicode (ISO/IEC 10646) character whose value is the hexadecimal value HHHH (one to four hex digits)

\UHHHHHHHH the Unicode (ISO/IEC 10646) character whose value is the hexadecimal value HHHHHHHH (one to eight hex digits)

\cx a control-x character

The expanded result is single-quoted, as if the dollar sign had not been present.


[1] You can, however, embed actual newlines in '...' and "..." strings; i.e., you can define strings that span multiple lines.

mklement0
  • 382,024
  • 64
  • 607
  • 775
16

From http://www.linuxtopia.org/online_books/bash_guide_for_beginners/sect_03_03.html:

Words in the form "$'STRING'" are treated in a special way. The word expands to a string, with backslash-escaped characters replaced as specified by the ANSI-C standard. Backslash escape sequences can be found in the Bash documentation.found

I guess it's forcing the script to escape the line feed to the proper ANSI-C standard.

Brad Swerdfeger
  • 911
  • 1
  • 8
  • 11
9

Re recovering the default IFS- this OLDIFS=$IFS is not necessary. Run new IFS in subshell to avoid overriding the default IFS:

ar=(123 321); ( IFS=$'\n'; echo ${ar[*]} )

Besides I don't really believe you recover the old IFS fully. You should double quote it to avoid line breaking such as OLDIFS="$IFS".

jnovack
  • 7,629
  • 2
  • 26
  • 40
Marek
  • 91
  • 1
  • 2
  • 2
    this is a really useful technique. i just used it for a cleaner shell join op: `args=$(IFS='&'; echo "$*")`. restoring `IFS` to `$' \t\n'` in a Bourne shell friendly manner is no mean feat. – jeberle Mar 11 '14 at 01:16
  • 1
    Re `Besides I don't really believe you recover the old IFS fully`: word splitting is _not_ performed on the RHS of variable assignments (but quote removal is), so `OLDIFS=$IFS` and `OLDIFS="$IFS"` behave the same way. – mklement0 Mar 13 '15 at 02:38
3

ANSI C-quoted strings is a key point. Thanks to @mklement0 .

You can test ANSI C-quoted strings with command od.

echo -n $'\n' | od -c
echo -n '\n' | od -c
echo -n $"\n" | od -c
echo -n "\n" | od -c

Outputs:

0000000  \n  
0000001

0000000   \   n   
0000002

0000000   \   n   
0000002

0000000   \   n   
0000002

You can know the meaning clearly by the outputs.

Big Shield
  • 612
  • 5
  • 15
2

Question:

What is the exact meaning of IFS=$'\n'?

Simple Answer:

Hey Bash! set the Internal Field Separator (IFS) to New Line


What is IFS ?

IFS is the character, Bash uses as word/item boundaries when processing character strings.

It is set to whitespace characters of space, tab, and newline, by default.

Example 1:

Use default value for IFS

string="first second:third forth:fifth"

for item in $string; do
    echo "$item"
done

Output:

first
second:third
forth:fifth

Example 2:

Set IFS to :

# Set the IFS to collon (:) character
IFS=:

string="first second:third forth:fifth"

for item in $string; do
    echo "$item"
done

Output:

first second  
third forth  
fifth
Peyman Mohamadpour
  • 17,954
  • 24
  • 89
  • 100
-8

It's like retrieving the value from a variable:

VAR='test'
echo VAR
echo $VAR

are different, so the dollar sign basically evaluates the content.

Pieter
  • 2,161
  • 1
  • 13
  • 5
  • 6
    This has nothing to do with variables. `$'FOO'` (unlike `$FOO` which this was question was not about) is a string literal. If you execute `echo $'VAR'`, you'll see that it prints the string `VAR`, not `test`. – sepp2k Nov 08 '10 at 21:43