Bash loop through directory including hidden file

Question

I am looking for a way to make a simple loop in bash over everything my directory contains, i.e. files, directories and links including hidden ones.

I will prefer if it could be specifically in bash but it has to be the most general. Of course, file names (and directory names) can have white space, break line, symbols. Everything but "/" and ASCII NULL (0×0), even at the first character. Also, the result should exclude the '.' and '..' directories.

Here is a generator of files on which the loop has to deal with :

#!/bin/bash
mkdir -p test
cd test
touch A 1 ! "hello world" \$\"sym.dat .hidden " start with space" $'\n start with a newline' 
mkdir -p ". hidden with space" $'My Personal\nDirectory'

So my loop should look like (but has to deal with the tricky stuff above):

for i in * ;
  echo ">$i<"
done

My closest try was the use of ls and bash array, but it is not working with, is:

IFS=$(echo -en "\n\b")
l=( $(ls -A .) )
for i in ${l[@]} ; do
echo ">$i<"
done
unset IFS

Or using bash arrays but the ".." directory is not exclude:

IFS=$(echo -en "\n\b")
l=( [[:print:]]* .[[:print:]]* )
for i in ${l[@]} ; do
echo ">$i<"
done
unset IFS

Answer 1

* doesn't match files beginning with ., so you just need to be explicit:

for i in * .[^.]*; do
    echo ">$i<"
done

.[^.]* will match all files and directories starting with ., followed by a non-. character, followed by zero or more characters. In other words, it's like the simpler .*, but excludes . and ... If you need to match something like ..foo, then you might add ..?* to the list of patterns.

Answer 2

As chepner noted in the comments below, this solution assumes you're running GNU bash along with GNU find GNU sort...

GNU find can be prevented from recursing into subdirectories with the -maxdepth option. Then use -print0 to end every filename with a 0x00 byte instead of the newline you'd usually get from -print.

The sort -z sorts the filenames between the 0x00 bytes.

Then, you can use sed to get rid of the dot and dot-dot directory entries (although GNU find seems to exclude the .. already).

I also used sed to get read of the ./ in front of every filename. basename could do that too, but older systems didn't have basename, and you might not trust it to handle the funky characters right.

(These sed commands each required two cases: one for a pattern at the start of the string, and one for the pattern between 0x00 bytes. These were so ugly I split them out into separate functions.)

The read command doesn't have a -z or -0 option like some commands, but you can fake it with -d "" and blanking the IFS environment variable.

The additional -r option prevents a backslash-newline combo from being interpreted as a line continuation. (A file called backslash\\nnewline would otherwise be mangled to backslashnewline.) It might be worth seeing if other backslash-combos get interpreted as escape sequences.

remove_dot_and_dotdot_dirs()
{
    sed \
      -e 's/^[.]\{1,2\}\x00//' \
      -e 's/\x00[.]\{1,2\}\x00/\x00/g'
}

remove_leading_dotslash()
{
    sed \
      -e 's/^[.]\///' \
      -e 's/\x00[.]\//\x00/g'
}

IFS=""
find . -maxdepth 1 -print0 |
  sort -z |
  remove_dot_and_dotdot_dirs |
  remove_leading_dotslash |
  while read -r -d "" filename
  do
      echo "Doing something with file '${filename}'..."
  done

Answer 3

Try the find command, something like:

find .

That will list all the files in all recursive directories.

To output only files excluding the leading . or .. try:

find . -type f -printf %P\\n

Answer 4

It may not be the most favorable way but I tried bellow thing

while read line ; do echo $line; done <<< $(ls -a | grep -v -w ".")

check the below trail which I did