0

Here's an example of my problematic code:

#!/bin/bash
fileList='fileList.txt'
#IFS=$'\n'
while read filename
do
  echo listing "$filename"
  ls -ligG "$filename"
done < "$fileList"
echo "done."
#unset IFS
exit 0

The output is:

listing /some/long/path/README.TXT
ls: cannot access /some/long/pa
: No such file or directoryDME.TXT

Notice that ls cuts off the path. Also notice that the end of the path/filename is appended to the error message (after "No such file or directory").

I just tested it with a path exactly this long and it still gives the error:

/this/is/an/example/of/shorter/name.txt

Anyone know what's going on? I've been messing with this for hours already :-/

In response to torek's answer, here is more info:

First, here's the modified script based on torek's suggestions:

#!/bin/bash
fileList=/settings/Scripts/fileList.txt
while IFS=$'\n' read -r filename
do
  printf 'listing %q\n' "$filename"
  ls -ligG $filename
done < "$fileList"
echo "done."
exit 0

Here's the output of that:

# ./test.sh
listing $'/example/pathname/myfile.txt\r'
: No such file or directorypathname/myfile.txt
done.

Notice there is some craziness going on still.

Here's the file. It does exist.

ls -ligG /example/pathname/myfile.txt
106828 -rwxrwx--- 1 34 Mar 28 00:55 /example/pathname/myfile.txt
MountainX
  • 6,217
  • 8
  • 52
  • 83
  • check the file permissions. Something like this is probably just your terminal displaying the long path incorrectly, `ls` is likely still getting the proper name. – Cfreak Mar 28 '12 at 04:42
  • haha - check the file permissions??? that's exactly what my script is failing to do. If `ls` was getting the proper name, it wouldn't be giving the error above. However, I will say that I can run the commands in the terminal outside of my simple script and they execute properly. But the script fails. – MountainX Mar 28 '12 at 04:51
  • I just produced the error on a file with these permissions: `-rwxrwx---` and I'm running my script as root anyway. – MountainX Mar 28 '12 at 04:53
  • Yes, the files exist and if I execute the commands inside the while-loop by pasting them into the terminal, each line executes as expected. But my script fails when reading the file list in the loop. – MountainX Mar 28 '12 at 05:10
  • Thanks to torek and FatalError, I found the source of the problem. It was Libre Office Calc. I'm pasting file names into my text file from a spreadsheet. When I copy them from Calc, a \r gets appended to the name. I haven't seen that in the past with Open Office. I'm using Kubuntu 12.04 now. And I don't see a setting in Calc to change this behavior. – MountainX Mar 28 '12 at 05:43

2 Answers2

2

That embedded newline is a clue: the error message should read ls: cannot access /some/long/path/README.TXT: No such file or directory (no newline after the "a" in "path"). Even if there were some mysterious truncation happening, the colon should happen right after the "a" in "path". It doesn't, so, the string is not what it seems to be.

Try:

printf 'listing %q\n' "$filename"

for printing the file name before invoking ls. Bash's built-in printf has a %q format that will quote funny characters.

I'm not sure what the intent of the commented-out IFS-setting is. Perhaps you want to prevent read from splitting at whitespace? You can put the IFS= in front of the read, and you might want to use read -r as well:

while IFS=$'\n' read -r filename; do ...; done < "$fileList"
torek
  • 448,244
  • 59
  • 642
  • 775
  • I implemented your suggestions and ran more tests. I edited my question to include that new info. – MountainX Mar 28 '12 at 05:22
  • Yep, the output shows that (as @FatalError suggested) the file names have carriage returns at the end. You can use tr to delete them, or, you can set IFS to eat carriage returns: `IFS=$'\n\r'`. – torek Mar 28 '12 at 05:30
  • Thank you. Your answer and FatalError were both helpful. Wish I could select both as the accepted answer. – MountainX Mar 28 '12 at 05:33
  • Well, he narrowed it down to the particular funny character. Mine is sort of a more general "how to figure out what the funny character(s) is/are". :-) – torek Mar 28 '12 at 05:38
2

Based on the unusual behavior, I'm going to say the file has CRLF line terminators. Your file names actually have an invisible carriage return appended to the name. In echo, this doesn't show up, since it just jumps to the first column then prints a newline. However, ls tries to access the file including the hidden carriage return, and in its error message, the carriage return causes the error message partially overwrite your path.

To trim these chars away, you can use tr:

tr -d '\r' < fileList.txt > fileListTrimmed.txt

and try using that file instead.

FatalError
  • 52,695
  • 14
  • 99
  • 116
  • Here's what I did: `rm fileList.txt; nano filelist.txt` insert a single filename and save; run my script. It works now. Funny thing is that I was editing the fileList in KWrite and I had blown away the list many times already. But KWrite was doing something very strange to the file. I'm not sure what. KWrite is set to use UNIX line terminators. I double checked that just now. So it was not CRLF line terminators, but it was a bad file in some way. Thanks. – MountainX Mar 28 '12 at 05:31
  • It was a line terminator problem. Libre Office Calc was sticking them onto the file names and they were persisting when I saved the text file with KWrite. – MountainX Mar 28 '12 at 05:46