0

I'm hoping to do a command on each line listed within foo.txt, where each line of foo.txt is a file name.

There's been plenty of great support for this question, and have tried a while read, another while read, I am now trying to do a for loop. However, I'm starting to think the issue is in the body of the loop.

#!/bin/bash
File=/mnt/d/R_projects/EC/foo.txt
Lines=$(cat $File)
for Line in $Lines
do
    echo  "fastp -i /mnt/d/R_projects/EC/download/fastq/$Line -o /mnt/e/EC/fastp_trimmed/$Line"
   ./fastp -i /mnt/d/R_projects/EC/download/fastq/$Line -o /mnt/e/EC/fastp_trimmed/$Line
done

I unfortunately receive the error: ERROR: Failed to open file: /mnt/d/R_projects/EC/download/fastq/SRR6132950_1.fastq

The file exists, and doing less confirms. Oddly, the echo doesn't echo what I was expecting and instead states: " -o /mnt/e/EC/fastp_trimmed/SRR6132950_1.fastqRR6132950_1.fastq"

What could be causing this issue? It's as if the first half was cut off.

OatMeal
  • 45
  • 6
  • 2
    See: [Looping through the content of a file in Bash](https://stackoverflow.com/q/1521462/3776858) – Cyrus May 08 '21 at 04:18
  • 1
    `Lines=$(cat $File)` is a *UUOc* (*Unnecessary Use Of* `cat`). Redirection is the proper approach. Before you can redirect a line and read it in a for-loop line-by-line, you need to set `IFS` to `'\n'`. Then you can do `for i in $(< somefile); do ...` – David C. Rankin May 08 '21 at 05:19
  • 3
    **Your file probably has carriage returns** (i.e. CRLF instead of LF) because you copied, accessed or editted it from Windows. Remove them; there are hundreds of existing Qs on this. – dave_thompson_085 May 08 '21 at 05:28
  • The canonical question about DOS/Windows files (with CRLF line endings) is: [Are shell scripts sensitive to encoding and line endings?](https://stackoverflow.com/questions/39527571/are-shell-scripts-sensitive-to-encoding-and-line-endings) You can also trim CRs in a `while read` loop (in bash, but not all other shells), with `while IFS=$'\r' read Line; do ... done <"$File"` – Gordon Davisson May 08 '21 at 07:45
  • In case you're dealing with carriage return characters, I hope you know you can use the `dos2unix` command? :-) – Dominique May 08 '21 at 09:01

1 Answers1

-1

Thank you all,

I had suspected the windows \r\n was causing an issue here, but I tried in nano to delete these and moved on. Notepad++ showed my error and was able to fix with notepad++ edit>EOL conversion>Linux (LF).

I also adapted the suggestions to remove cat and add IFS - the suggestion was helpful.

#!/bin/bash
IFS=$'\n'
for Line in $(< /mnt/d/R_projects/EC/foo.txt); do
    echo  "$Line"
    echo  "fastp -i /mnt/d/R_projects/EC/download/fastq/$Line -o /mnt/e/EC/fastp_trimmed/$Line"
   ./fastp -i /mnt/d/R_projects/EC/download/fastq/$Line -o /mnt/e/EC/fastp_trimmed/$Line
done

I have like a million needs for running commands based on a text file and this will be extremely helpful for further applications.

OatMeal
  • 45
  • 6
  • This still won't work if any of the lines could be interpreted as a glob, as the unquoted command substitution will undergo filename expansion as well as word-splitting. [Use a while loop](http://mywiki.wooledge.org/BashFAQ/001). – chepner May 08 '21 at 13:26