Bash function can indent piped lines multiple times without waiting for whole input

Question

What I want to achieve:

Define a function that can be used to pipe input, such as echo input | my_function
This function modifies every inputted line, e.g., it adds indentation at the beginning.
This function can be reused twice and in that case does its modification (double indent) twice, e.g. echo input | my_function | my_function results in \t\tinput.
This function does not wait for whole input to be supplied, it can print out the line directly without seeing all input.

As for my test, please see the following script:

#!/usr/bin/env bash

main() {
  echo 'first:'
  echo 'once' | tab_indent_to_right
  echo 'twice' | tab_indent_to_right | tab_indent_to_right
  {
    echo 'wait 2 sec'
    sleep 2
    echo 'wait 2 sec'
    sleep 2
    echo 'waited'
  } | tab_indent_to_right
}

tab_indent_to_right() {
  # while read -r line; do echo $'\t'"$line"; done  #  double indent not working
  # awk -v prefix='\t' '{print prefix $0}'          #  buffer not working
  # sed 's/^/\t/'                                   #  buffer not working
  # xargs -I {} echo $'\t{}'                        #  double indent not working
  # xargs -L1 echo $'\t'                            #  double indent not working
}

main

Each line in tab_indent_to_right is my failed attempt to solve the problem. They have two different problems, either:

Double indent is not working e.g., tab_indent_to_right | tab_indent_to_right
Or, the lines are not being flushed/printed directly but instead being buffered. In other words, the function waits for all sleeps and prints out everything at once, instead of printing the lines as they come.

How can I create this function so both double calls gives me the modification I want and also the script does not wait for complete execution of the piped shell?

See [How to make output of any shell command unbuffered?](https://stackoverflow.com/q/3465619/4154375). The mechanisms described in the answers could be used to with `sed` or `awk`. — pjh, Dec 11 '22 at 01:21

score 6 · Accepted Answer · answered Dec 10 '22 at 23:25

6

I suggest to set IFS to nothing to keep spaces/tabs.

tab_indent_to_right() {
  while IFS= read -r line; do echo $'\t'"$line"; done
}

answered Dec 10 '22 at 23:25

Cyrus

84,225
14
89
153

Gordon Davisson · Answer 2 · 2022-12-10T23:39:52.883

In recentish versions of awk, you can use the fflush() function to force it to send output lines immediately:

awk -v prefix='\t' '{print prefix $0; fflush()}'

Also, depending on the version of sed you have, you may be able to add either the -u flag (for "unbuffered"; GNU sed supports this) or the -l flag ("line buffered"; bsd sed supports this). If you have some other version of sed... check its man page to see if it has a similar option.

BTW, the bsd version of sed doesn't support \t for tab; but if you use $'s/^/\t/', bash will convert \t to tab before passing it to sed. This works with both versions of sed, but not with shells that don't support ANSI quoting mode (i.e. dash).

Bash function can indent piped lines multiple times without waiting for whole input

2 Answers2