How to store NUL output of a program in bash script?

Question

Suppose there is a directory 'foo' which contains several files:

ls foo:
1.aa 2.bb 3.aa 4.cc

Now in a bash script, I want to count the number of files with specific suffix in 'foo', and display them, e.g.:

SUFF='aa'
FILES=`ls -1 *."$SUFF" foo`
COUNT=`echo $FILES | wc -l`
echo "$COUNT files have suffix $SUFF, they are: $FILES"

The problem is: if SUFF='dd', $COUNT also equal to 1. After google, the reason I found is when SUFF='dd', $FILES is an empty string, not really the null output of a program, which will be considered to have one line by wc. NUL output can only be passed through pipes. So one solution is:

COUNT=`ls -1 *."$SUFF" foo | wc -l`

but this will lead to the ls command being executed twice. So my question is: is there any more elegant way to achieve this?

the file-names end with `foo`? Can you give one such sample name? Do you want to recursively search in sub-directories also? — Inian, Sep 28 '16 at 10:12
foo is the directory. file-names end with aa,bb,cc. Not need to search sub-directories. — xlwang, Sep 28 '16 at 10:15

score 2 · Accepted Answer · answered Sep 28 '16 at 10:16

2

$ shopt -s nullglob
$ FILES=(*)
$ echo "${#FILES[@]}"
4
$ FILES=(*aa)
$ echo "${#FILES[@]}"
2
$ FILES=(*dd)
$ echo "${#FILES[@]}"
0
$ SUFFIX=aa
$ FILES=(*"$SUFFIX")
$ echo "${#FILES[@]}"
2
$ SUFFIX=dd
$ FILES=(*"$SUFFIX")
$ echo "${#FILES[@]}"
0

answered Sep 28 '16 at 10:16

Ignacio Vazquez-Abrams

776,304
153
1,341
1,358

Wow great explanation! – 123 Sep 28 '16 at 10:23
3

@123: That's funny, I didn't write one. Have you visited an ophthalmologist recently? – Ignacio Vazquez-Abrams Sep 28 '16 at 10:24
Is there a simple way to output contents of FILES one by one? – xlwang Sep 28 '16 at 10:30
http://stackoverflow.com/questions/8880603/loop-through-array-of-strings-in-bash-script – Ignacio Vazquez-Abrams Sep 28 '16 at 10:33
This is the earliest and best answer till now. So the answer is essentially not using programs like ls or find, but using pathname expansion built in bash. – xlwang Sep 28 '16 at 11:07

redneb · Answer 2 · 2016-09-28T10:33:19.823

0

If you only need the file count, I would actually use find for that:

find '/path/to/directory' -mindepth 1 -maxdepth 1 -name '*.aa' -printf '\n' | wc -l

This is more reliable as it handles correctly filenames with line breaks. The way this works is that find outputs one empty line for each matching file.

Edit: If you want to keep the file list in an array, you can use a glob:

GLOBIGNORE=".:.."
shopt -s nullglob
FILES=(*aa)
COUNT=${#arr[@]}
echo "$COUNT"

edited Sep 28 '16 at 10:33

answered Sep 28 '16 at 10:13

redneb

21,794
6
42
54

But I also want to output the names of these files later in the script. – xlwang Sep 28 '16 at 10:18
1

@Inian What do you mean? – redneb Sep 28 '16 at 10:22
@Inian It does because they only output a newline for each filename, not the actual filename – 123 Sep 28 '16 at 10:26
@Inian yes but print0 is only on gnu as well. – 123 Sep 28 '16 at 10:30
@123: My bad! thought it was a `POSIX` standard, guess only `-print` is portable – Inian Sep 28 '16 at 10:32
This uses GNU extensions to `find` that simulate the POSIX shell solution `for i in /path/to/directory/*.aa; do [ ! -e "$i" ] && break; echo; done`. – chepner Sep 28 '16 at 11:41

Mustafa DOGRU · Answer 3 · 2016-09-28T10:31:04.410

0

you can also try this;

#!/bin/bash
SUFF='aa'
FILES=`ls -1 *."$SUFF" foo`
FILENAMES=`echo $FILES | awk -F ':' '{print $2}'`
COUNT=`echo $FILENAMES | wc -w`
echo "$COUNT files have suffix $SUFF, they are: $FILENAMES"

if inserted echo $FILES in your script, output is foo: 1.aa 2.aa 3.aa so

awk -F ':' '{print $2}' gets 1.aa 2.aa 3.aa from $FILES variable

wc -w prints the word counts

edited Sep 28 '16 at 10:31

answered Sep 28 '16 at 10:24

Mustafa DOGRU

3,994
1
16
24

score 0 · Answer 4 · 2016-09-28T11:34:51.577

0

The reason is that the option nullglob is unset by default in bash:

If no matching file names are found, and the shell option nullglob is not enabled, the word is left unchanged. If the nullglob option is set, and no matches are found, the word is removed.

So, just set the nullglob option, and run you code again:

shopt -s nullglob
SUFF='aa'
FILES="$(printf '%s\n' foo/*."$SUFF")"
COUNT="$(printf '%.0s\n' foo/*."$SUFF" | wc -l)"
echo "$COUNT files have suffix $SUFF, they are: $FILES"

Or better yet:

shopt -s nullglob
suff='aa'
files=( foo/*."$suff" )
count=${#file[@]}
echo "$count files have suffix $suff, they are: ${files[@]}"

edited Sep 28 '16 at 11:34

answered Sep 28 '16 at 10:32

Thx. But if file names contain \n, it may fail. – xlwang Sep 28 '16 at 10:57
@xlwang Ok, solved. The `%.0s\n` prints only a new line per each file. – Sep 28 '16 at 11:29

How to store NUL output of a program in bash script?

4 Answers4