Iteration over strings contain white spaces

Question

I'm doing a simple iteration over a remote text file content, but get stuck in splitting file names as they have spaces inside.

#!/bin/bash
for u in $(curl http://XXXXXX.com/urls.txt)
do
    wget "${u%|*}" -O "${u#*|}" -P ./Directory/
done

Text file contains below strings per line:

https://OOOOOOOO.com?url1|My file with spaces in name.mp4
https://OOOOOOOO.com?url2|How to escape spaces?.mp4
https://OOOOOOOO.com?url3|Mixed up.mp4
https://OOOOOOOO.com?url4|Asking StackOverFlow!.mp4

I looked at many similar issues already, made many approaches but all results the same thing.

When I do a echo ${u#*|} alone, I have the output:

My
file
with
spaces
in
name.mp4
Mixed
up.mp4
.
.
.

What's wrong?

score 2 · Accepted Answer · answered Nov 05 '14 at 11:48

2

A while-read loop would be simpler here:

curl http://XXXXXX.com/urls.txt |
while IFS='|' read -r url filename; do
    wget "${url}" -O "${filename}" -P ./Directory/
done

answered Nov 05 '14 at 11:48

glenn jackman

238,783
38
220
352

score 1 · Answer 2 · answered Nov 05 '14 at 09:50

1

use Input Field Separator (IFS)

This variable determines how Bash recognizes fields, or word boundaries, when it interprets character strings. LINK

e.g:

IFS=$'\n'
for u in $(cat file)
do
   echo ${u#*|}
done

OUTPUT:

My file with spaces in name.mp4
How to escape spaces?.mp4
Mixed up.mp4
Asking StackOverFlow!.mp4

answered Nov 05 '14 at 09:50

Baba

852
1
17
31

Thanks! I used `IFS` for newlines before, but splitting was done with `read` and not these shell globs! Would you mind to say what's wrong with mine? and what's the `$` after `IFS=`? – revo Nov 05 '14 at 09:55
1

@revo : read this: http://stackoverflow.com/questions/4128235/bash-shell-scripting-what-is-the-exact-meaning-of-ifs-n – Baba Nov 05 '14 at 10:05
2

Don't mess with IFS on the read. Just do `while read -d '|' url fname; do whatever; done <$filename` That will save the first part in `url` and the second in `fname`. The just quote `"$fname"` when you use it. – David C. Rankin Nov 05 '14 at 10:06
1

@DavidC.Rankin Would you add your comment as an answer? – revo Nov 05 '14 at 10:25
Be glad to, but Glen already posted the equivalent. More than happy to let him have the credit :) – David C. Rankin Nov 05 '14 at 19:55

Iteration over strings contain white spaces

2 Answers2