How to cat multiple files from a list of files in Bash?

Question

I have a text file that holds a list of files. I want to cat their contents together. What is the best way to do this? I was doing something like this but it seems overly complex:

let count=0
while read -r LINE
do
    [[ "$line" =~ ^#.*$ ]] && continue
    if [ $count -ne 0 ] ; then
        file="$LINE"
        while read PLINE
        do
            echo $PLINE | cat - myfilejs > /tmp/out && mv /tmp/out myfile.js
        done < $file
    fi
    let count++
done < tmp

I was skipping commented lines and running into issues. There has to be a better way to do this, without two loops. Thanks!

score 19 · Accepted Answer · answered Jul 23 '12 at 19:59

19

Or in a simple command

cat $(grep -v '^#' files) > output

answered Jul 23 '12 at 19:59

Bernhard

3,619
1
25
50

2

@CharlesDuffy I doubt that for this particular application one would really use space-separated filenames. Rather propose an improvement, as in my opinion a oneliner is a nicer solution than a 8-line script. Are you actually using filenames with whitespace EVER? – Bernhard Jul 25 '12 at 12:42
5

@Bernhard Don't take this the wrong way, but *of course* we deal with filenames with whitespace all the time. We support users who don't know the difference, and why should they if it's easy enough to write a script that can handle it? – kojiro Jul 25 '12 at 12:47
3

And if didn't have comments: `cat $(cat files) > output` – mustafa.0x Dec 31 '13 at 10:21

Ciro Santilli OurBigBook.com · Answer 2 · 2021-05-21T17:22:10.140

xargs cat < files

The advantage of xargs over $(cat) is that cat expands to a huge list of arguments which could fail if you have a lot of files in the list due to Linux' maximum command line length.

Example without caring about #:

printf 'a\nb\nc\n' > files
printf '12\n3\n' > a
printf '4\n56\n' > b
printf  '8\n9\n' > c
xargs cat < files

Output:

More specific example ignoring # as requested by OP:

printf 'a\nb\n#c\n' > files
printf '12\n3\n' > a
printf '4\n56\n' > b
printf  '8\n9\n' > c
grep -v '^#' files | xargs cat

Output:

Related: How to pipe list of files returned by find command to cat to view all the files

score 5 · Answer 3 · answered Jul 23 '12 at 20:04

5

#!/bin/bash

files=()
while read; do
    case "$REPLY" in
        \#*|'') continue;;
        *) files+=( "$REPLY" );;
    esac
done < input
cat "${files[@]}"

What's better about this approach is that:

The only external command, cat, only gets executed once.
It's pretty careful to maintain significant whitespace for any given line/filename.

answered Jul 23 '12 at 20:04

kojiro

74,557
19
143
201

In comparison to this answer: cat $(grep -v '^#' files) > output Are there any obvious advantages? – user1470511 Jul 23 '12 at 20:07
3

@user1470511 Yes, the `grep` answer can break if any of the filenames have whitespace in them. – kojiro Jul 23 '12 at 20:13

Ignacio Vazquez-Abrams · Answer 4 · 2012-07-23T20:02:46.360

3

{
  while read file
  do
    #process comments here with continue
    cat "$file"
  done
} < tmp > newfile

edited Jul 23 '12 at 20:02

answered Jul 23 '12 at 19:54

Ignacio Vazquez-Abrams

776,304
153
1,341
1,358

I fixed the use of the variable `file`. Is there a real need for the braces? – chepner Jul 23 '12 at 19:59
@chepner: No, since the variable use is unambiguous. – Ignacio Vazquez-Abrams Jul 23 '12 at 20:02

score 1 · Answer 5 · answered Jul 23 '12 at 19:56

1

How about cat $(cat listoffiles) | grep -v "^#"?

answered Jul 23 '12 at 19:56

Lars Kotthoff

107,425
16
204
204

1

Don't you mean `cat $(grep -v '^#' listoffiles)`? What about whitespace issues? – kojiro Jul 23 '12 at 19:58
OK, as long as none of the filenames contain spaces. – chepner Jul 23 '12 at 20:00
1

@chepner Or as long as whitespaces are escaped in `listoffiles`, or names quoted. – loxaxs Jan 09 '17 at 18:52

score 0 · Answer 6 · edited Mar 20 '17 at 15:30

0

I ended up with this:

sed '/^$/d;/^#/d;s/^/cat "/;s/$/";/' files | sh > output

Steps:

sed removes blank lines & commented out lines from files.
sed then wraps each line in cat " and ";.
Script is then piped to sh with and into output.

edited Mar 20 '17 at 15:30

David Makogon

69,407
21
141
189

answered Mar 20 '17 at 15:12

sh3rmy

1

I forgot to mention that this crudely handles whitespaces by wrapping the file names in quotes. – sh3rmy Mar 20 '17 at 22:18

How to cat multiple files from a list of files in Bash?

6 Answers6

Linked

Related