2

I created a shell script to convert all wave files to mp3 files. I use Ubuntu 18.04, FFmpeg was installed using apt, and I run my script in bash.

My script:

#!/bin/bash

ls *.wav | while read file
do
    echo $file
    ffmpeg -i "$file" -codec:a libmp3lame -b:a 192k "${file%.*}.mp3"
done

The target files are followings: (include space characters. The real filenames are longer, but I simplified the filenames. Also in this case, the problem occurs)

% ls *.wav
'01 A.wav'  '02 A.wav'  '03 A.wav'

The problem is that sometimes $file in the loop is blank or a part of filename strangely ('echo $file' shows that), and ffmpeg says '[broken filename]: No such file or directory'. I confirmed the following things.

  • When I comment out the ffmpeg line, 'echo' shows what I expected. ($file not broken)
  • When I replace ffmpeg to a similar command like 'lame', it works. ($file not broken)
  • When I replace 'ls *.wav | while read file' to 'for file in *.wav', it works. ($file not broken)

So, only when I use combination of 'ls' and 'ffmpeg', $file is broken. What's going on? Or do I misunderstand something?

user3288408
  • 67
  • 5
  • Now I was able to reproduce the problem. It did not work with empty files, but it worked with actual `.wav` files. For me `read` reads `2 A.wav` instead of `02 A.wav` resulting in the `ffmpeg` error `2 A.wav: No such file or directory`. – Socowi Dec 13 '18 at 09:14
  • Thank you for the try and additional information. It's really mysterious. – user3288408 Dec 13 '18 at 09:19
  • Yeah, I'm also out if ideas. First I suspected some sort of race condition, then that `ffmpeg` modifies `stdin` of `read`. However both theories do not make any sense and working around them with `sleep 1` and `> /dev/null` does not resolve the issue. – Socowi Dec 13 '18 at 09:31
  • Maybe you should edit your question to make it more clear that it's about understanding what happens and not fixing the loop with `for file in *.wav`. Change the title to something that does *not* contain `while | read ls` (red flag for experienced bash users). Start off with something like *»I know that `ls | while read` is bad because …«*. Use `declare -p file` instead of `echo $file`. Also you may remove the spaces from the filenames. For me the problem also happens with the files `1.wav`, `2.wav`, `3.wav`, `4.wav`. File `2` and `4` are read as just `.wav` instead of `N.wav`. – Socowi Dec 13 '18 at 09:35
  • @Socowi I can't reproduce the problem. Which version of bash and coreutils are you guys using? – oguz ismail Dec 13 '18 at 09:41
  • @oguzismail I'm on arch linux with `coreutils 8.30-1`, `bash 4.4.023-1`, `ffmpeg 1:4.1-1`. – Socowi Dec 13 '18 at 09:45
  • This might be of interest: https://github.com/damianociarla/node-ffmpeg/issues/21 – cdarke Dec 13 '18 at 09:45
  • @cdarke The problem also occurs for filenames without spaces. I can reproduce the problem with relative pathnames `1.wav` and `2.wav` in `/tmp/sub/`. – Socowi Dec 13 '18 at 09:46
  • @Socowi we're using the same versions. brb gonna try on Ubuntu 18.04 – oguz ismail Dec 13 '18 at 09:48
  • 1
    @oguzismail But maybe different locales and `wav` files. I can reproduce the problem with `export LC_ALL=C` and files generated by `ffmpeg -f lavfi -i anullsrc -t 1 1.wav; echo {2..4}.wav | xargs -n 1 cp 1.wav`. – Socowi Dec 13 '18 at 09:59
  • 1
    @user3288408 See: https://stackoverflow.com/a/44249993/10248678 – oguz ismail Dec 13 '18 at 10:07
  • @Socowi the guy deserves a prize but his answer isn't even accepted. this is so sad :( – oguz ismail Dec 13 '18 at 10:11
  • 1
    Yes, think so too. But man, I was so close. `ffmpeg` *was* meddling with `stdin`, but I assumed it would add somthing and tried `> /dev/null` when it actually consumed something and I should have used `< /dev/null`. – Socowi Dec 13 '18 at 10:13
  • @Socowi Maybe next time :D `<&-` works too – oguz ismail Dec 13 '18 at 10:24
  • Sorry, now I'm back and everything was over.. Thank you for discussion and information. I couldn't find the same question. Since the problem was solved, I'll close this question. Thank you again! – user3288408 Dec 13 '18 at 11:43

1 Answers1

2

You should not use ls to pipe a list of files, because this approach will not work well for corner cases like filenames with blanks.

The most readable way is to use globbing:

for file in *.wav
do
  echo "$file"
  ffmpeg -i "$file" -codec:a libmp3lame -b:a 192k "${file%.*}.mp3"
done

find also does a great job here, but you will end up with a little less maintainable code:

find -name *.wav -exec ffmpeg -i {} -codec:a libmp3lame -b:a 192k {}.mp3 \;

The find example will give you filenames ending in .wav.mp3 instead of .mp3, if you want to avoid that, you have to call a shell from find, allowing you to either use basename or do something like ${file.*} (you would have to assign find's {} to a variable like file in that shell first:

find -name "*.wav" -exec sh -c "file=\"{}\" ; ffmpeg -i \"\$file\" -codec:a libmp3lame -b:a 192k \${file%.*}.mp3" \;
Michael Jaros
  • 4,586
  • 1
  • 22
  • 39
  • While this may solve the problem, it does not answer the question. OP already mentioned in the question that using `for file in *.wav` resolves the issues. – Socowi Dec 13 '18 at 09:06
  • Thank you for the informative answer and the comment. Also I'm curious about what causes this mysterious behavior. I also welcome answers about this phenomenon. – user3288408 Dec 13 '18 at 09:25
  • @Socowi That is correct, unfortunately I missed that info while scanning the question earlier. I hope I'll get to looking into it again a little later. – Michael Jaros Dec 13 '18 at 09:50
  • @Michael Jaros I appreciate your answer. Sorry for the misleading question... – user3288408 Dec 13 '18 at 11:45
  • @user3288408 Ah, that's not your fault, I did not read properly :-) – Michael Jaros Dec 13 '18 at 21:57