0

I am running a script that has been working fine. However, yesterday, I got a couple errors. These errors are after several loops of the script: 

sed: cant read file3.txt: No such file or directory
grep: file3.txt: No such file or directory
grep: file3.txt: No such file or directory
sed: cant read file3.txt: No such file or directory
grep: file3.txt: No such file or directory

Keep in mind, these errors do not happen consistently. It's occurring once in a while somewhere near this part of the script. File3.txt is the file not being found:

cat file1.txt | while read LINE; do grep -m 1 $LINE file2.txt >> file3.txt; done
sed -i 's/string//g'  file3.txt
grep 'string' file3.txt | cut -d '|' -f1-2 > file4.txt
grep -v 'string' file3.txt | cut -d '|' -f1-2 >> file5.txt
sed -i 's/string//' file3.txt
grep -Fvf file3.txt file1.txt > file6.txt

Now, I'm thinking that since file3.txt is being appended, or later operated on by SED, sometimes the next command starts too soon and it can't find the file? Should I put a wait command in between?

I have looked up many pages with this error, but was unable to find anything:

If you think that putting a wait or sleep command will help, please let me know. Or, if you think there's a better solution, that would be great too. I'm running on Cygwin terminal. Any insight is greatly appreciated.

Community
  • 1
  • 1
DomainsFeatured
  • 1,426
  • 1
  • 21
  • 39
  • Did you intend to have a blank in the file name from the first 'sed'? – phreed Sep 28 '16 at 15:38
  • You said this script gets run repeatedly. Does file3.txt get deleted between runs? Is is possible for the two runs to be concurrent? I am wondering about the 'sed -i'. – phreed Sep 28 '16 at 15:44
  • 1
    Whenever I see a whole bunch of temporary files, with multiple usages of grep and sed, I think of awk. I don't know why what you've got isn't working but I am almost certain that what you're trying to do can be reduced to a single awk script. If you show us your input and your desired output, then we can help you more effectively. – Tom Fenech Sep 28 '16 at 15:45
  • 2
    If `file1.txt` is ever empty then `file3.txt` won't be created. – ccarton Sep 28 '16 at 15:48
  • 1
    I would not think a wait or sleep would help. The commands should not "start too soon". I would look for something in file1.txt that should be escaped when inserted into the grep command as $LINE. I think your file3.txt is not being created due to a quote or a semicolon or something. – bmb Sep 28 '16 at 15:48
  • @phreed, Yes, file3.txt gets `rm -f` right before the first command shown here, I can add it but don't think it would make a difference. Secondly, they are not running concurrently or in parallel. – DomainsFeatured Sep 28 '16 at 15:49
  • @TomFenech, You're probably right. However, we need the output files for various uses so it has to be broken up that way. Thanks for the input though. – DomainsFeatured Sep 28 '16 at 15:50
  • @ccarton, `file1.txt` would not be empty. However, that would be a good catch. I think we're on the right track. – DomainsFeatured Sep 28 '16 at 15:51
  • How do you get 4 grep errors trying to read file3.txt but only 3 greps in your script? – phreed Sep 28 '16 at 15:51
  • @ccarton that's a good point. Note that if the redirection was moved out of the loop `done >> file3.txt` (or possibly just `>`, depending on the context), this wouldn't happen. – Tom Fenech Sep 28 '16 at 15:52
  • @bmb, thanks for answering my initial question. I think this would make sense, if something is in `file1.txt` that needs to be escaped, then it could caused `file3.txt` not be generated. Would changing the quoting or bracket of $LINE to "{$LINE}" help that? The only character that is special in `file1.txt` is a dash `-`. – DomainsFeatured Sep 28 '16 at 15:55
  • Wait a second, @ccarton, I think you nailed it. I'm looking at previous jobs, maybe it didn't happen multiple times. It could have happened just once. And the errors was only from one job. I think you solved it! – DomainsFeatured Sep 28 '16 at 15:59
  • 2
    Glad I could help. bmb's observation should also be addressed even if it's not causing problems now. I think it should probably say `grep -F -m1 "$LINE" ...` – ccarton Sep 28 '16 at 16:05
  • Thanks, will do change that as well. If you would like to provide an answer post, I will select it as correct and upvote :-) Please do the same :-) – DomainsFeatured Sep 28 '16 at 16:06

3 Answers3

3

Instead of redirecting to file3.txt inside the while loop, redirect the whole loop. Then the file will be created even if the loop never runs because the input file is empty.

while read LINE; do 
    grep -m 1 $LINE file2.txt
done < file1.txt > file3.txt
Barmar
  • 741,623
  • 53
  • 500
  • 612
1

If file1.txt is ever empty then file3.txt won't be created.

ccarton
  • 3,556
  • 16
  • 17
1

Also do grep -m 1 $LINE file2.txt will cause problems if there are crucial characters (space is the easiest of them).
Let's assume that the $LINE variable contains more than one word separated by spaces: hello world.
Now the command looks like this: grep -m 1 hello world file2.txt - grep interpretation will look something like this: let's find all hello in file named world and file named file2.txt in current folder.

Using "$LINE" instead of $LINE will lead you to a whole different scenario.
Look at the difference between the following two: 

grep -m 1 $LINE file2.txt
grep -m 1 "$LINE" file2.txt
zx485
  • 28,498
  • 28
  • 50
  • 59
run4gnu
  • 333
  • 1
  • 2
  • 7