Looping through directories that have space in their name

Question

I am sorry if someone asked this question gain, but I didn't find a solution which worked for me.

I wrote a small bash script, which works with normal folders. It should run a command for every Subfolder. But if there is a space in a Folder Name, there is a problem.

If there is a Folder called "New Folder" in /share/Download, the script will try to access to the folder "/share/Download/New" (instead of "New Folder")

i also tried with echo "$d"

    for d in `find /share/Download -mindepth 1 -type d`
do
    echo
    echo $d
    curl http://localhost:2345/api/command -X POST -d '{"name": "something", "path": "'"$d"'"}' --header "X-Api-Key:123456789"
    echo
done

This could help: https://unix.stackexchange.com/questions/9496/looping-through-files-with-spaces-in-the-names — Robert Seaman, Apr 17 '18 at 22:50
I was about to say the same thing. Use `IFS` to tell when to break lines. In this case, add `IFS=$'\n'` before the `for` loop. — Prav, Apr 17 '18 at 22:58
Also read [BashFAQ #20: "How can I find and safely handle file names containing newlines, spaces or both?"](http://mywiki.wooledge.org/BashFAQ/020) — Gordon Davisson, Apr 17 '18 at 23:04
If you don't need to recursively search subfolders, `for d in /share/Download/*/; do` will suffice. — chepner, Apr 17 '18 at 23:08
@PraveenP, that's buggy -- it'll still expand globs, and directories can have newlines in their names. If you have a directory created with `mkdir '*'`, for example, then even with `IFS=$'\n'`, iterating over it will then iterate over *all contents* of that directory, including non-directory files not returned by `find` at all. — Charles Duffy, Apr 17 '18 at 23:14
@CharlesDuffy I though Unix don't allow folder or file name with special characters such as `\n`? Which turn into `n`. — Prav, Apr 17 '18 at 23:18
@PraveenP, incorrect -- they are allowed (in general; some filesystems may be more restrictive, but ext3/ext4/btrfs/etc on Linux permit them with no problem). Run `mkdir $'\n'` to test. Changing `\n` into `n` is a side effect of `read` without `-r`, or using an incorrect quoting type. — Charles Duffy, Apr 17 '18 at 23:19
@CharlesDuffy I see what you mean now. Thanks for clarifying that. — Prav, Apr 17 '18 at 23:22
Possible duplicate of [Loop over directories with whitespace in Bash](https://stackoverflow.com/q/4895484/608639). Also see [Bash loop command through list containing spaces](https://stackoverflow.com/q/36173381/608639) — jww, Oct 06 '18 at 08:09

codeforester · Accepted Answer · 2018-04-17T23:01:16.203

Shell is doing word splitting on find's output and that is the reason you are having the trouble.

Use process substitution to properly read the output of find:

while read -r d; do
  # your logic
done < <(find /share/Download -mindepth 1 -type d)

Even better, make find emit NUL terminated output so that directories that have a newline in them would be handled appropriately as well:

while read -r -d '' d; do
  # your logic
done < <(find /share/Download -mindepth 1 -type d -print0)

This answer can help you further.

score 0 · Answer 2 · answered Apr 17 '18 at 23:06

0

I allready found the problem and a solution after more research.

for d in isn't whitespace save.

The following code did the trick:

    find  /share/Download -mindepth 1 -type d|while read fname; do
  curl http://localhost:1234/api/command -X POST -d '{"name": "something", "path": "'"$fname"'"}' --header "X-Api-Key:123456789"
done

answered Apr 17 '18 at 23:06

wizzi

23
4

Needs to be `while IFS= read -r fname` to correctly handle names that start and end with spaces or contain backslashes. And even that's buggy, because names with backslashes or literal double quotes will generate invalid JSON. If you want to generate content that's guaranteed to be correctly-escaped JSON, use `jq`. – Charles Duffy Apr 17 '18 at 23:15
1

`curl ... -d "$(jq -c --arg fname "$fname" '{"name": "something", "path": $fname}')"` is guaranteed to generate well-formed JSON, no matter what your filenames are. And see the advice from codeforester about using `-print0` and `IFS= read -r -d '' fname` – Charles Duffy Apr 17 '18 at 23:16

Looping through directories that have space in their name

2 Answers2