0

I encountered ls: write error: Broken pipe when I run ls -l | head in a directory containing too many files. I did not encounter that problem when running ls | head. I just wonder why -l option would cause that problem. Also I would like to know how to resolve that problem but still with long information of files printed.

To reproduce my issue, you can first open a within terminal provided by Jupyter Notebook,

enter image description here

and then run the following codes.

cd
mkdir test
cd test
for i in {1..200}; do touch $i; done
ls -l | head

Output is as follows.

total 0
-rw-rw-r-- 1 s1855106 s1855106 0 Apr 14 10:14 1
-rw-rw-r-- 1 s1855106 s1855106 0 Apr 14 10:14 10
-rw-rw-r-- 1 s1855106 s1855106 0 Apr 14 10:14 100
-rw-rw-r-- 1 s1855106 s1855106 0 Apr 14 10:14 101
-rw-rw-r-- 1 s1855106 s1855106 0 Apr 14 10:14 102
-rw-rw-r-- 1 s1855106 s1855106 0 Apr 14 10:14 103
-rw-rw-r-- 1 s1855106 s1855106 0 Apr 14 10:14 104
-rw-rw-r-- 1 s1855106 s1855106 0 Apr 14 10:14 105
-rw-rw-r-- 1 s1855106 s1855106 0 Apr 14 10:14 106
ls: write error: Broken pipe
Hans Lub
  • 5,513
  • 1
  • 23
  • 43
Fei Yao
  • 1,502
  • 10
  • 10
  • 1
    [Why *not* parse `ls`?](http://unix.stackexchange.com/questions/128985/why-not-parse-ls) – Cyrus Apr 14 '20 at 09:20
  • 1
    I cannot reproduce this, but have a look at [How can I fix a broken pipe error](https://superuser.com/questions/554855/how-can-i-fix-a-broken-pipe-error). Possibly your shell (and hence any command started by it) is ignoring SIGPIPE (as an illustration, `(trap '' PIPE; ls -l | head)` will reliably reproduce your issue). You can see the signal disposition of your `bash` shell with the builtin `trap -p` – Hans Lub Apr 14 '20 at 09:50
  • I can't reproduce this error - I tried with 20,000 files, but `ls -l | head` works as expected - could be something to do with your shell (bash v5?) or your ls command (gnu ls?) ? – jared_mamrot Apr 14 '20 at 09:53
  • Thank you all for your very helpful comments! I have now found that I will encounter this problem if I use the terminal provided by Jupyter Notebook. All the things are fine if I use other terminal like Git Bash. I have edited my problem accordingly and I appreciate your further help. I am using Jupyter Notebook Terminal mainly because I want to do all the things on web pages. – Fei Yao Apr 14 '20 at 10:52

1 Answers1

0

Python (at least older versions of python2) doesn't reset SIGPIPE handling for subprocesses

This means that any commands started by it (e.g. commands you start in a Jupyter notebook) will ignore SIGPIPE, exactly as if you had issued trap '' PIPE. This, in turn, means that commands in a pipeline don't catch the SIGPIPE that is sent to them when they try to write to their stdout while the following command (tail, in your case) in the pipeline has finished, and thus see a write error instead. See, for example, this bug report.

(trap '' PIPE; ls -l | head) will reliably reproduce your problem in any (non-Jupyter) shell.

Apparently, python3 has fixed this, so you could try changing the python version used by your notebook. Alternatively, you could try "un-ignoring" SIGPIPE by issuing trap - PIPE

Hans Lub
  • 5,513
  • 1
  • 23
  • 43