Stack Overflow
Stack Overflow

Questions tagged [pipe]

A pipe is an interprocess connection between file descriptors of two processes. A pipe is created with the POSIX pipe() function (from ). Shells create pipes between processes if the "|" symbol is used: "cmd1 | cmd2" directs the output of cmd1 to the input of cmd2. On Windows use CreatePipe(). This mechanism redirects standard input, standard output, and standard error into the calling process in .NET and Java.

Every program that runs on the command line has three data streams connected to it: STDIN(0) - standard input, STDOUT (1) - standard output - data printed by the program, STDERR(2) - standard error.

There are ways to connect streams between programs and files called piping and redirections.

Piping is a mechanism for sending data from one program to another using the "|" operator. The operator feeds the output from the program on the left as input to the program on the right.

Example 

$ cat two_columns
column1:cloth
column2:strawberries
column3:fish
column4:chocolate
column5:punch cards
$ cat two_columns | awk -F: '{print $1}'
column1
column2
column3
column4
column5 
$ cat two_columns | awk -F: '{print "HAS: " $2}'
HAS: cloth
HAS: strawberries
HAS: fish
HAS: chocolate
HAS: punch cards

Useful Links

9809 questions
1241
votes
11 answers

How can I pipe stderr, and not stdout?

I have a program that writes information to stdout and stderr, and I need to process the stderr with grep, leaving stdout aside. Using a temporary file, one could do it in two steps: command > /dev/null 2> temp.file grep 'something' temp.file But…
user80168
828
votes
7 answers

How to redirect Windows cmd stdout and stderr to a single file?

I'm trying to redirect all output (stdout + stderr) of a Windows command to a single file: C:\>dir 1> a.txt 2> a.txt The process cannot access the file because it is being used by another process. Is it possible, or should I just redirect to two…
ripper234
  • 222,824
  • 274
  • 634
  • 905
516
votes
16 answers

Pipe output and capture exit status in Bash

I want to execute a long running command in Bash, and both capture its exit status, and tee its output. So I do this: command | tee out.txt ST=$? The problem is that the variable ST captures the exit status of tee and not of command. How can I…
flybywire
  • 261,858
  • 191
  • 397
  • 503
373
votes
8 answers

How to use `subprocess` command with pipes

I want to use subprocess.check_output() with ps -A | grep 'process_name'. I tried various solutions but so far nothing worked. Can someone guide me how to do it?
zuberuber
  • 3,851
  • 2
  • 16
  • 20
339
votes
6 answers

How can I detect if my shell script is running through a pipe?

How do I detect from within a shell script if its standard output is being sent to a terminal or if it's piped to another process? The case in point: I'd like to add escape codes to colorize output, but only when run interactively, but not when…
user111422
331
votes
7 answers

Retrieving the output of subprocess.call()

How can I get the output of a process run using subprocess.call()? Passing a StringIO.StringIO object to stdout gives this error: Traceback (most recent call last): File "", line 1, in File…
Jeffrey Aylesworth
  • 8,242
  • 9
  • 40
  • 57
302
votes
5 answers

How to pipe stdout while keeping it on screen ? (and not to a output file)

I would like to pipe standard output of a program while keeping it on screen. With a simple example (echo use here is just for illustration purpose) : $ echo 'ee' | foo ee <- the output I would like to see I know tee could copy stdout to file but…
gentooboontoo
  • 4,653
  • 3
  • 20
  • 15
256
votes
17 answers

Read values into a shell variable from a pipe

I am trying to get bash to process data from stdin that gets piped into, but no luck. What I mean is none of the following work: echo "hello world" | test=($(< /dev/stdin)); echo test=$test test= echo "hello world" | read test; echo…
ldog
  • 11,707
  • 10
  • 54
  • 70
255
votes
15 answers

How to pipe list of files returned by find command to cat to view all the files

I am doing a find to get a list of files. How do I pipe it to another utility like cat so that cat displays the contents of all those files? Afterwards, I'd use grep on that to search some text in those files.
Devang Kamdar
  • 5,617
  • 8
  • 24
  • 16
244
votes
4 answers

Piping command output to tee but also save exit code of command

I have a shell script in which I wrap a command (mvn clean install), to redirect the output to a logfile. #!/bin/bash ... mvn clean install $@ | tee $logfile echo $? # Does not show the return code of mvn clean install Now if mvn clean install…
Wolkenarchitekt
  • 20,170
  • 29
  • 111
  • 174
242
votes
19 answers

Why does cURL return error "(23) Failed writing body"?

It works ok as a single tool: curl "someURL" curl -o - "someURL" but it doesn't work in a pipeline: curl "someURL" | tr -d '\n' curl -o - "someURL" | tr -d '\n' it returns: (23) Failed writing body What is the problem with piping the cURL output?…
static
  • 8,126
  • 15
  • 63
  • 89
238
votes
2 answers

How to use `jq` in a shell pipeline?

I can't seem to get jq to behave "normally" in a shell pipeline. For example: $ curl -s https://api.github.com/users/octocat/repos | jq | cat results in jq simply printing out its help text*. The same thing happens if I try to redirect jq's…
mgalgs
  • 15,671
  • 11
  • 61
  • 74
209
votes
5 answers

What are the parameters for the number Pipe - Angular 2

I have used the number pipe below to limit numbers to two decimal places. {{ exampleNumber | number : '1.2-2' }} I was wondering what the logic behind '1.2-2' was? I have played around with these trying to achieve a pipe which filters to zero…
rushtoni88
  • 4,417
  • 7
  • 22
  • 31
206
votes
8 answers

How to open every file in a folder

I have a python script parse.py, which in the script open a file, say file1, and then do something maybe print out the total number of characters. filename = 'file1' f = open(filename, 'r') content = f.read() print filename, len(content) Right…
B.Mr.W.
  • 18,910
  • 35
  • 114
  • 178
200
votes
4 answers

Multiprocessing - Pipe vs Queue

What are the fundamental differences between queues and pipes in Python's multiprocessing package? In what scenarios should one choose one over the other? When is it advantageous to use Pipe()? When is it advantageous to use Queue()?
Jonathan Livni
  • 101,334
  • 104
  • 266
  • 359
1
2 3
…
99 100