88

There are often times I will grep -n whatever file to find what I am looking for. Say the output is:

1234: whatev 1
5555: whatev 2
6643: whatev 3

If I want to then just extract the lines between 1234 and 5555, is there a tool to do that? For static files I have a script that does wc -l of the file and then does the math to split it out with tail & head but that doesn't work out so well with log files that are constantly being written to.

vvvvv
  • 25,404
  • 19
  • 49
  • 81
  • 6
    See http://stackoverflow.com/questions/83329/how-can-i-extract-a-range-of-lines-from-a-text-file-on-unix – GreenMatt May 26 '10 at 15:14

7 Answers7

126

Try using sed as mentioned on http://linuxcommando.blogspot.com/2008/03/using-sed-to-extract-lines-in-text-file.html. For example use

sed '2,4!d' somefile.txt

to print from the second line to the fourth line of somefile.txt. (And don't forget to check http://www.grymoire.com/Unix/Sed.html, sed is a wonderful tool.)

vvvvv
  • 25,404
  • 19
  • 49
  • 81
Scorchio
  • 2,763
  • 2
  • 20
  • 28
  • 6
    One useful follow up bit of information is how to prepend the line numbers onto the sed result... pipe it into nl like so: `sed ''"$start"','"$end"'!d' somefile.txt | nl -ba -v$start` – phyatt Nov 07 '16 at 18:16
  • @Scorchio What does `!d` mean? – Manuel Jordan Jan 24 '22 at 19:43
  • 2
    @ManuelJordan `d` is the delete command of sed. `!` reverses the restriction (ie. in this case, the specified range). So `2,4!d` means dropping everything except lines 2-4. – Scorchio Feb 15 '22 at 11:54
  • @Scorchio thanks for the explanation. - Normally `!` go in the beginning - something like `!2,4d` – Manuel Jordan Feb 15 '22 at 12:09
  • Does it delete the other lines from the file or just from the standard output? – princess_hacker Jul 19 '22 at 11:28
  • 1
    @princess_hacker Sed doesn't touch the original file in itself. It just outputs the filtered and transformed parts of the file. – Scorchio Sep 17 '22 at 15:47
49

The following command will do what you asked for "extract the lines between 1234 and 5555" in someFile.

sed -n '1234,5555p' someFile
vvvvv
  • 25,404
  • 19
  • 49
  • 81
javaPlease42
  • 4,699
  • 7
  • 36
  • 65
13

If I understand correctly, you want to find a pattern between two line numbers. The awk one-liner could be

awk '/whatev/ && NR >= 1234 && NR <= 5555' file

You don't need to run grep followed by sed.

Perl one-liner:

perl -ne 'if (/whatev/ && $. >= 1234 && $. <= 5555) {print}' file
Andrew
  • 3,733
  • 1
  • 35
  • 36
Mark Lakata
  • 19,989
  • 5
  • 106
  • 123
6

Line numbers are OK if you can guarantee the position of what you want. Over the years, my favorite flavor of this has been something like this:

sed "/First Line of Text/,/Last Line of Text/d" filename

which deletes all lines from the first matched line to the last match, including those lines.

Use sed -n with "p" instead of "d" to print those lines instead. Way more useful for me, as I usually don't know where those lines are.

psbrightly
  • 61
  • 1
  • 3
2

Put this in a file and make it executable:

#!/usr/bin/env bash
start=`grep -n $1 < $3 | head -n1 | cut -d: -f1; exit ${PIPESTATUS[0]}`
if [ ${PIPESTATUS[0]} -ne 0 ]; then
    echo "couldn't find start pattern!" 1>&2
    exit 1
fi
stop=`tail -n +$start < $3 | grep -n $2 | head -n1 | cut -d: -f1; exit ${PIPESTATUS[1]}`
if [ ${PIPESTATUS[0]} -ne 0 ]; then
    echo "couldn't find end pattern!" 1>&2
    exit 1
fi

stop=$(( $stop + $start - 1))

sed "$start,$stop!d" < $3

Execute the file with arguments (NOTE that the script does not handle spaces in arguments!):

  1. Starting grep pattern
  2. Stopping grep pattern
  3. File path

To use with your example, use arguments: 1234 5555 myfile.txt

Includes lines with starting and stopping pattern.

Janus Troelsen
  • 20,267
  • 14
  • 135
  • 196
1

If I want to then just extract the lines between 1234 and 5555, is there a tool to do that?

There is also ugrep, a GNU/BSD grep compatible tool but one that offers a -K option (or --range) with a range of line numbers to do just that:

ugrep -K1234,5555 -n '' somefile.log

You can use the usual GNU/BSD grep options and regex patterns (but it also offers a lot more such as -K.)

Dr. Alex RE
  • 1,772
  • 1
  • 15
  • 23
0

If you want lines instead of line ranges, you can do it with perl: eg. if you want to get line 1, 3 and 5 from a file, say /etc/passwd:

perl -e 'while(<>){if(++$l~~[1,3,5]){print}}' < /etc/passwd
dagelf
  • 1,468
  • 1
  • 14
  • 25
  • 2
    FYI, That `$l` is "dollar el" not "dollar one". A more perlish (i.e. shorter) command is `perl -ne 'if($.~~[1,3,5]){print}' /etc/passwd`. – Mark Lakata Jul 07 '16 at 16:41