68

I have some set of bash functions which output some information:

  • find-modelname-in-epson-ppds
  • find-modelname-in-samsung-ppds
  • find-modelname-in-hp-ppds
  • etc ...

I've been writing functions which read output and filter it:

function filter-epson {
    find-modelname-in-epson-ppds | sed <bla-blah-blah>
}

function filter-hp {
    find-modelname-in-hp-ppds | sed <the same bla-blah-blah>
}
etc ...

But the I thought that it would be better do something like this:

function filter-general {
    (somehow get input) | sed <bla-blah-blah>
}

and then call in another high-level functions:

function high-level-func {
    # outputs filtered information
    find-modelname-in-hp/epson/...-ppds | filter-general 
}

How can I achieve that with the best bash practices?

likern
  • 3,744
  • 5
  • 36
  • 47
  • If you're looking for good practice, you can already replace `function fname` with `fname()`. – gniourf_gniourf Dec 22 '12 at 17:18
  • 1
    What's the difference? Only the brevity? But I think definition with `function` looks more expressive, isn't it? – likern Dec 22 '12 at 17:22
  • 1
    What do you mean by _more expressive_? In [this link](http://wiki.bash-hackers.org/scripting/obsolete) you'll see that it's obsolete and deprecated (and not POSIX). – gniourf_gniourf Dec 22 '12 at 17:25
  • 1
    What do your functions `find-modelname-...` really look like and do? Maybe you should tell us more so that we can advice the best option. You're clearly trying to factor some pieces of code, but we need to know what it is exactly. – gniourf_gniourf Dec 22 '12 at 17:34
  • 1
    @gniourf_gniourf the link you reference says `function fname { ... }` is fine. What's deprecated is `function fname() {...}` – bames53 Oct 03 '13 at 15:54
  • @bames53 `function` [is not defined by POSIX](http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_09_05) – gniourf_gniourf Oct 03 '13 at 16:59
  • 1
    @gniourf_gniourf Yes, but it's not deprecated in bash as you indicated. Anyway the paren syntax is uglier and doesn't make sense; They aren't used to call the function and one doesn't pass parameters inside them. – bames53 Oct 03 '13 at 17:09
  • @bames53 ok, then use it as much as you want `;)`. – gniourf_gniourf Oct 03 '13 at 17:15

4 Answers4

87

If the question is How do I pass stdin to a bash function?, then the answer is:

Shellscript functions take stdin the ordinary way, as if they were commands or programs. :)

input.txt:

HELLO WORLD
HELLO BOB
NO MATCH

test.sh:

#!/bin/sh

myfunction() {
    grep HELLO
}

cat input.txt | myfunction

Output:

hobbes@metalbaby:~/scratch$ ./test.sh 
 HELLO WORLD 
 HELLO BOB 

Note that command line arguments are ALSO handled in the ordinary way, like this:

test2.sh:

#!/bin/sh

myfunction() {
    grep "$1"
}

cat input.txt | myfunction BOB

Output:

hobbes@metalbaby:~/scratch/$ ./test2.sh 
 HELLO BOB 
daveloyall
  • 2,140
  • 21
  • 23
  • This does not work on bash 4.1.0. I get the output: " HELLO WORLD \n HELLO BOB \n NO MATCH" – DocSalvager Dec 05 '14 at 10:43
  • Ok, it's because of my usage of `\n` in the input string. I've edited the answer; it now just cats an input file, for the sake of simplicity. This question is about accessing `stdin` from a function--and that part works even in bash 2.05b. :) – daveloyall Dec 05 '14 at 19:10
  • 1
    Ah, yes... because echo does not support escape sequences. Another way to handle that is define a variable like NL="" and replace "\n" with "$NL". Working now. Thanks! – DocSalvager Dec 06 '14 at 08:11
  • This makes no sense for a multi-command function. E.g., assign a variable and then invoke a command that takes stdin following. Or what if I want to pass the stdin to two different functions (à la `tee`)? The lack of explanation here makes this answer virtually useless if you don't already know the answer and have an even *slightly* more complex case. – jpmc26 Aug 15 '18 at 21:39
  • 3
    jpmc26, hi, Welcome to Stack Overflow. I hope this answer isn't really 'virtually useless'. The question is about the simple case, so the answer is about the simple case, too. I've never tried to pipe `stdin` to a function that contains more than one command. I'll play with that idea in my shell later. Meanwhile, if you have time, please post a new question about that. – daveloyall Aug 16 '18 at 14:53
36

To be painfully explicit that I'm piping from stdin, I sometimes write

cat - | ...
glenn jackman
  • 238,783
  • 38
  • 220
  • 352
  • 1
    In my case, indeed I needed to be "painfully explicit", otherwise it didn't work. – Mahdi Oct 27 '14 at 10:29
  • @Mahdi What do you mean? Are there shells out there which require the use of `cat -` in this case/context? Which? When? How? – 7heo.tk May 21 '15 at 23:30
  • I have a script that runs as a mapper in a Hadoop map-reduce job, and the input contents are given to me via stdin. I needed to read the whole input into a file for further processing, and this solution worked for me. – Mahdi May 24 '15 at 12:13
  • 2
    This should be an accepted answer. It is short, simple and most importantly, straight to the point. Basically, man page for `cat` recommends doing it just like that. Or just `cat | `, but that is less explicit. – Sevastyan Savanyuk Feb 02 '19 at 20:38
31

A very simple means to get stdin into a variable is to use read. By default, it reads file descriptor "0", i.e. stdin i.e., /dev/stdin.

Example Function:

input(){ local in; read in; echo you said $in; }                    

Example implementation:

echo "Hello World" | input               

Result:

you said Hello World

Additional info

You don't need to declare a variable as being local, of course. I just included that for the sake of good form. Plain old read in does what you need.

So you understand how read works, by default it reads data off the given file descriptor (or implicit stdin) and blocks until it encounters a newline. Much of the time, you'll find that will implicitly be attached to your input, even if you weren't aware of it. If you have a function that seems to hang with this mechanism just keep this detail in mind (there are other ways of using read to deal with that...).

More robust solutions

Adding on to the basic example, here's a variation that lets you pass the input via a stdin OR an argument:

input()
{ 
    local in=$1; if [ -z "$in" ]; then read in; fi
    echo you said $in
}

With that tweak, you could ALSO call the function like:

input "Hello World"

How about handling an stdin option plus other arguments? Many standard nix utilities, especially those which typically work with stdin/stdout adhere to the common practice of treating a dash - to mean "default", which contextually means either stdin or stdout, so you can follow the convention, and treat an argument specified as - to mean "stdin":

input()
{ 
    local a=$1; if [ "$a" == "-" ]; then read a; fi
    local b=$2
    echo you said $a $b
}

Call this like:

input "Hello" "World"

or

echo "Hello" | input - "World"

Going even further, there is actually no reason to only limit stdin to being an option for only the first argument! You might create a super flexible function that could use it for any of them...

input()
{ 
    local a=$1; if [ "$a" == "-" ]; then read a; fi
    local b=$2; if [ "$b" == "-" ]; then read b; fi
    echo you said $a $b
}

Why would you want that? Because you could formulate, and pipe in, whatever argument you might need...

myFunc | input "Hello" -

In this case, I pipe in the 2nd argument using the results of myFunc rather than the only having the option for the first.

BuvinJ
  • 10,221
  • 5
  • 83
  • 96
  • Coming to this answer from the future - your example will only read the first line of multi-line input streamed to the function - is there a better way to grab 'all 'of stdin and use it in a single variable? – Cinderhaze Jul 18 '23 at 03:57
  • Hi future reader! Use the `-d` option for `read` to specify a different delimiter instead of the default newline. See: https://linuxcommand.org/lc3_man_pages/readh.html – BuvinJ Jul 18 '23 at 13:19
  • 1
    I'm not sure what you're attempting to do exactly, but if reading a file it would probably make sense to use `cat` instead of `read`. If gathering directly typed input, you could prompt the user to end their text in some special character to denote the end of input. – BuvinJ Jul 18 '23 at 13:26
  • See: https://serverfault.com/a/783506 – BuvinJ Jul 18 '23 at 13:29
  • I ultimately ended up using cat - I was writing a 'logging library' for some of our bash scripts, and wanted a 'log_debug' type of method that could either take an argument OR have the result piped in. – Cinderhaze Aug 11 '23 at 05:08
6

Call sed directly. That's it.

function filter-general {
    sed <bla-blah-blah>
}
John Kugelman
  • 349,597
  • 67
  • 533
  • 578