Bash, grep between two lines with specified string

Question

Example:

a43
test1
abc
cvb
bnm
test2
kfo

I need all lines between test1 and test2. Normal grep does not work in this case. Do you have any propositions?

Answer 1

Print from test1 to test2 (Trigger lines included)

awk '/test1/{f=1} /test2/{f=0;print} f'
awk '/test1/{f=1} f; /test2/{f=0}' 
awk '/test1/,/test2/'

test1
abc
cvb
bnm
test2

---

Prints data between test1 to test2 (Trigger lines excluded)

awk '/test1/{f=1;next} /test2/{f=0} f' 
awk '/test2/{f=0} f; /test1/{f=1}' 

abc
cvb
bnm

Answer 2

You could use sed:

sed -n '/test1/,/test2/p' filename

In order to exclude the lines containing test1 and test2, say:

sed -n '/test1/,/test2/{/test1/b;/test2/b;p}' filename

Answer 3

If you can only use grep:

grep -A100000 test1 file.txt | grep -B100000 test2 > new.txt

grep -A and then a number gets the lines after the matching string, and grep -B gets the lines before the matching string. The number, 100000 in this case, has to be large enough to include all lines before and after.

If you don't want to include test1 and test2, then you can remove them afterwards by grep -v, which prints everything except the matching line(s):

egrep -v "test1|test2" new.txt > newer.txt

or everything in one line:

grep -A100000 test1 file.txt | grep -B100000 test2 | egrep -v "test1|test2" > new.txt 

Answer 4

Yep, normal grep won't do this. But grep with -P parameter will do this job.

$ grep -ozP '(?s)test1\n\K.*?(?=\ntest2)' file
abc
cvb
bnm

\K discards the previously matched characters from printing at the final and the positive lookahead (?=\ntest2) asserts that the match must be followed by a \n newline character and then test2 string.

Answer 5

The following script wraps up this process. More details in this similar StackOverflow post

get_text.sh

function show_help()
{
  HELP=$(doMain $0 HELP)
  echo "$HELP"
  exit;
}

function doMain()
{
  if [ "$1" == "help" ]
  then
    show_help
  fi
  if [ -z "$1" ]
  then
    show_help
  fi
  if [ -z "$2" ]
  then
    show_help
  fi

  FILENAME=$1
  if [ ! -f $FILENAME ]; then
      echo "File not found: $FILENAME"
      exit;
  fi

  if [ -z "$3" ]
  then
    START_TAG=$2_START
    END_TAG=$2_END
  else
    START_TAG=$2
    END_TAG=$3
  fi

  CMD="cat $FILENAME | awk '/$START_TAG/{f=1;next} /$END_TAG/{f=0} f'"
  eval $CMD
}

function help_txt()
{
HELP_START
  get_text.sh: extracts lines in a file between two tags

  usage: FILENAME {TAG_PREFIX|START_TAG} {END_TAG}

  examples:
    get_text.sh 1.txt AA     => extracts lines in file 1.txt between AA_START and AA_END
    get_text.sh 1.txt AA BB  => extracts lines in file 1.txt between AA and BB
HELP_END
}

doMain $*

Answer 6

To make it more deterministic and not having to worry about size of file, use the wc -l and cut the output.

grep -Awc -l test.txt|cut -d" " -f1 test1 test.txt | grep -Bwc -l test.txt|cut -d" " -f1 test2

To make it easier to read, assign it to a variable first.

fsize=wc -l test.txt|cut -d" " -f1; grep -A$fsize test1 test.txt | grep -B$fsize test2

Answer 7

You can do something like this too. Lets say you this file test.txt with content:

a43
test1
abc
cvb
bnm
test2
kfo

You can do

cat test.txt | grep -A10 test1 | grep -B10 test2

where -A<n> is to get you n lines after your match in the file and -B<n> is to give you n lines before the match. You just have to make sure that n > number of expected lines between test1 and test2. Or you can give it large enough to reach EOF.

Result:

test1
abc
cvb
bnm
test2

Answer 8

The answer by PratPor above:

cat test.txt | grep -A10 test1 | grep -B10 test2

is cool.. but if you don't know the file length:

cat test.txt | grep -A1000 test1 | grep -B1000 test2

Not deterministic, but not too bad. Anyone have better (more deterministic)?