86

I want to get a list of files and then read the results into an array where each array element corresponds to a file name. Is this possible?

vhs
  • 9,316
  • 3
  • 66
  • 70
dublintech
  • 16,815
  • 29
  • 84
  • 115
  • 1
    Yes, it is possible. Maybe not advisable if the names might contain arbitrary characters (spaces and newlines in the names cause grief), but it is doable. Which bit of the manual did you have difficulty understanding? – Jonathan Leffler Jun 11 '12 at 13:53
  • How is the _list_ being defined? bash has arrays, but depending on how the list is generated, different techniquest are better than others. In any case, post also your own attempts to solve the problem. – user1934428 Oct 26 '21 at 11:57

6 Answers6

135

Don't use ls, it's not intended for this purpose. Use globbing.

shopt -s nullglob
array=(*)
array2=(file*)
array3=(dir/*)

The nullglob option causes the array to be empty if there are no matches.

Dennis Williamson
  • 346,391
  • 90
  • 374
  • 439
  • Thanks, Is there anyway I can pipe the results of these? I tried something like arr(* | grep ".txt") but it does not like it. – dublintech Jun 11 '12 at 14:00
  • 11
    @dublintech: You don't need `grep`, just include the string in your glob: `array=(*.txt)` or `array=(*foo*)` – Dennis Williamson Jun 11 '12 at 14:02
  • 1
    You can also [append filenames](https://stackoverflow.com/a/1951523/712334) to an array, [output filenames](https://stackoverflow.com/a/15692004/712334) and [loop through](https://stackoverflow.com/a/8880633/712334) them. – vhs Aug 04 '18 at 06:08
  • To my pleasant surprise, this also works with filenames that have whitespace or special characters in them! Tested with GNU bash, version 5.0. – famzah Jan 02 '23 at 10:04
32

Following will create an array arr with ls output in current directory:

arr=( $(ls) )

Though using output of ls is not safe at all.

Much better and safer than ls you can use echo *:

arr=( * )

echo ${#arr[@]} # will echo number of elements in array

echo "${arr[@]}" # will dump all elements of the array
anubhava
  • 761,203
  • 64
  • 569
  • 643
  • 10
    `ls` is not necessary and should not be used for this purpose. – Dennis Williamson Jun 11 '12 at 13:54
  • Agreed, I just wanted to tell how to create an array from some command output. However I edited my answer to stress that `ls` output should be avoided. – anubhava Jun 11 '12 at 13:56
  • 3
    When the array expansion is not quoted, all elements of the array are represented as a string, rather than individual elements of the array. Unquoted `${arr[*]}` and `${arr[@]}` are the same. – jordanm Jun 11 '12 at 16:19
  • 1
    Does anyone knows what is the max number of filenames / elements an array can hold? – dat789 Mar 31 '16 at 09:16
  • 3
    anubhava, please remove your `arr=( $(ls) )` line. Or if you want to leave it, please explicitly add a mention like **don't do this, it's broken.** – gniourf_gniourf Apr 01 '18 at 12:09
5

In bash you can create an array of filenames with pathname expansion (globbing) like so:

#!/bin/bash
SOURCE_DIR=path/to/source
files=(
   "$SOURCE_DIR"/*.tar.gz
   "$SOURCE_DIR"/*.tgz
   "$SOURCE_DIR"/**/*
)

The above will create an array called files and add to it N array elements, where each element in the array corresponds to an item in SOURCE_DIR ending in .tar.gz or .tgz, or any item in a subdirectory thereof with subdirectory recursion possible as Dennis points out in the comments.

You can then use printf to see the contents of the array including paths:

printf '%s\n' "${files[@]}" # i.e. path/to/source/filename.tar.gz

Or using parameter substitution to exclude the pathnames:

printf '%s\n' "${files[@]##*/}" # i.e. filename.tgz
vhs
  • 9,316
  • 3
  • 66
  • 70
  • 1
    You need to have `shopt -s globstar` in order to use recursive globbing (`**`). – Dennis Williamson Aug 04 '18 at 11:46
  • Not for me. And the default Bash on MacOS is 3.2 which does not have `globstar`. What does counting the characters in some directory names with `echo /usr/**/ | wc -c` output for you? On my Mac, in Sierra with Bash 3.2 or in Bash 4.4 with `globstar` off, it outputs 122. If I run Bash 4.4 with `globstar` on, I get 277904. The latter is clearly recursive and the former is not. BTW, `shopt` is defined, but I presume you mean `globstar` (in Bash 3.2, `shopt -p globstar` gives an error, in Bash 4.4 it shows whether it's set or unset). – Dennis Williamson Aug 04 '18 at 13:09
  • 1
    `hash` only works with external executables (try `hash -t ls` and `help hash`) and isn't going to show anything for `shopt` because it's a builtin or for `globstar` because it's an option rather than an executable. Try `type -a shopt` to show where `shopt` is coming from and `shopt` by itself to show the settings of all your options. Use `echo "$BASH_VERSION"` to show the version of your currently running shell (if it's Bash) and look at the output of `ps -o tty,command` to see if it's actually `/bin/bash`. Compare `find /usr -type d | wc -l` and `echo /usr/**/ | tr -cd " " | wc -c` ... – Dennis Williamson Aug 05 '18 at 12:36
  • ... the counts should be fairly close if recursive globbing is working. My counts are in the neighborhood of 3800. – Dennis Williamson Aug 05 '18 at 12:36
  • 1
    I'm ok with leaving it here. It's not specific to the OP's question, but it's a useful technique when outputting file names/paths. – Dennis Williamson Aug 05 '18 at 17:40
  • Yes `globstar` was added in Bash 4.0. I still don't understand how `hash shopt` does anything useful for you since `shopt` is a builtin. And `hash` isn't intended to show whether something is defined - it shows whether the path to an external executable has been cached (or manipulates the cache). To find whether (and how) something is defined, use `type -a`. Also don't use `which` or `whereis` since they are external and know nothing about builtins, functions and aliases. – Dennis Williamson Aug 06 '18 at 13:46
2

Actually, ls isn't the way to go. Try this:

declare -a FILELIST
for f in *; do 
    #FILELIST[length_of_FILELIST + 1]=filename
    FILELIST[${#FILELIST[@]}+1]=$(echo "$f");
done

To get a filename from the array use:

echo ${FILELIST[x]}

To get n filenames from the array starting from x use:

echo ${FILELIST[@]:x:n}

For a great tutorial on bash arrays, see: http://www.thegeekstuff.com/2010/06/bash-array-tutorial/

Pedro
  • 96
  • 5
xizdaqrian
  • 716
  • 6
  • 10
  • 2
    You should iterate over the file glob *OR* create an array using a file glob as in my answer - not *both*!. When adding elements to an array, there's no reason to use the length of the array in a complex index expression (it won't work as expected if the array is sparse, for one thing). `array+=(element)`. There's no reason to use `$(echo "$f")` just do the assignment directly. There's a missing closing curly brace on one of your `echo` statements. – Dennis Williamson Sep 30 '13 at 14:17
  • 2
    are you aware that what you're doing is a broken way to just do `FILELIST = ( * )`? (or, rather `FILELIST += ( * )`). Why on earth do you use `$(echo "$f")` instead of just `"$f"`? – gniourf_gniourf Apr 01 '18 at 12:12
1

Try this,

path="" # could set to any absolute path
declare -a array=( "${path}"/* )

I'm assuming you'll pull out the unwanted stuff from the list later.

gniourf_gniourf
  • 44,650
  • 9
  • 93
  • 104
KSU_Physics
  • 11
  • 1
0

If you need a more specific file listing that cannot be returned through globbing, then you can use process substitution for the find command with a while loop delimited with null characters.

Example:

files=()
while IFS= read -r -d $'\0' f; do
    files+=("$f")
done < <(find . -type f -name '*.dat' -size +1G -print0)
Snake
  • 501
  • 7
  • 9