unix - head AND tail of file

Question

Say you have a txt file, what is the command to view the top 10 lines and bottom 10 lines of file simultaneously?

i.e. if the file is 200 lines long, then view lines 1-10 and 190-200 in one go.

Answer 1

You can simply:

(head; tail) < file.txt

And if you need to uses pipes for some reason then like this:

cat file.txt | (head; tail)

Note: will print duplicated lines if number of lines in file.txt is smaller than default lines of head + default lines of tail.

Answer 2

ed is the standard text editor

$ echo -e '1+10,$-10d\n%p' | ed -s file.txt

Answer 3

(sed -u 10q; echo ...; tail) < file.txt

Just another variation on the (head;tail) theme, but avoiding the initial buffer fill issue for small files.

Answer 4

For a pure stream (e.g. output from a command), you can use 'tee' to fork the stream and send one stream to head and one to tail. This requires using either the '>( list )' feature of bash (+ /dev/fd/N):

( COMMAND | tee /dev/fd/3 | head ) 3> >( tail )

or using /dev/fd/N (or /dev/stderr) plus subshells with complicated redirection:

( ( seq 1 100 | tee /dev/fd/2 | head 1>&3 ) 2>&1 | tail ) 3>&1
( ( seq 1 100 | tee /dev/stderr | head 1>&3 ) 2>&1 | tail ) 3>&1

(Neither of these will work in csh or tcsh.)

For something with a little better control, you can use this perl command:

COMMAND | perl -e 'my $size = 10; my @buf = (); while (<>) { print if $. <= $size; push(@buf, $_); if ( @buf > $size ) { shift(@buf); } } print "------\n"; print @buf;'

Answer 5

Based on J.F. Sebastian's comment:

cat file | { tee >(head >&3; cat >/dev/null) | tail; } 3>&1

This way you can process first line and the rest differently in one pipe, which is useful for working with CSV data:

{ echo N; seq 3;} | { tee >(head -n1 | sed 's/$/*2/' >&3; cat >/dev/null) | tail -n+2 | awk '{print $1*2}'; } 3>&1

N*2
2
4
6

Answer 6

It took make a lot of time to end-up with this solution which, seems to be the only one that covered all use cases (so far):

command | tee full.log | stdbuf -i0 -o0 -e0 awk -v offset=${MAX_LINES:-200} \
          '{
               if (NR <= offset) print;
               else {
                   a[NR] = $0;
                   delete a[NR-offset];
                   printf "." > "/dev/stderr"
                   }
           }
           END {
             print "" > "/dev/stderr";
             for(i=NR-offset+1 > offset ? NR-offset+1: offset+1 ;i<=NR;i++)
             { print a[i]}
           }'

Feature list:

• live output for head (obviously that for tail is not possible)
• no use of external files
• progressbar one dot for each line after the MAX_LINES, very useful for long running tasks.
• progressbar on stderr, assuring that the progress dots are separated from the head+tail (very handy if you want to pipe stdout)
• avoids possible incorrect logging order due to buffering (stdbuf)
• avoid duplicating output when total number of lines is smaller than head + tail.

Answer 7

head -10 file.txt; tail -10 file.txt

Other than that, you'll need to write your own program / script.

Answer 8

the problem here is that stream-oriented programs don't know the length of the file in advance (because there might not be one, if it's a real stream).

tools like tail buffer the last n lines seen and wait for the end of the stream, then print.

if you want to do this in a single command (and have it work with any offset, and do not repeat lines if they overlap) you'll have to emulate this behaviour I mentioned.

try this awk:

awk -v offset=10 '{ if (NR <= offset) print; else { a[NR] = $0; delete a[NR-offset] } } END { for (i=NR-offset+1; i<=NR; i++) print a[i] }' yourfile

Answer 9

I have been looking for this solution for a while. Tried it myself with sed, but the problem with not knowing the length of file/stream beforehand was insurmountable. Of all the options available above, I like Camille Goudeseune's awk solution. He did make a note that his solution left extra blank lines in the output with a sufficiently small data set. Here I provide a modification of his solution that removes the extra lines.

headtail() { awk -v offset="$1" '{ if (NR <= offset) print; else { a[NR] = $0; delete a[NR-offset] } } END { a_count=0; for (i in a) {a_count++}; for (i=NR-a_count+1; i<=NR; i++) print a[i] }' ; }

Answer 10

Well, you can always chain them together. Like so, head fiename_foo && tail filename_foo. If that is not sufficient, you could write yourself a bash function in your .profile file or any login file that you use:

head_and_tail() {
    head $1 && tail $1
}

And, later invoke it from your shell prompt: head_and_tail filename_foo.

Answer 11

I wrote a simple python app to do this: https://gist.github.com/garyvdm/9970522

It handles pipes (streams) as well as files.

Answer 12

drawing on ideas above (tested bash & zsh)

but using an alias 'hat' Head and Tails

alias hat='(head -5 && echo "^^^------vvv" && tail -5) < '


hat large.sql

Answer 13

Consumes stdin, but simple and works for 99% of use cases

head_and_tail

#!/usr/bin/env bash
COUNT=${1:-10}
IT=$(cat /dev/stdin)
echo "$IT" | head -n$COUNT
echo "..."
echo "$IT" | tail -n$COUNT

example

$ seq 100 | head_and_tail 4
1
2
3
4
...
97
98
99
100

Answer 14

First 10 lines of file.ext, then its last 10 lines:

cat file.ext | head -10 && cat file.ext | tail -10

Last 10 lines of the file, then the first 10:

cat file.ext | tail -10 && cat file.ext | head -10

You can then pipe the output elsewhere too:

(cat file.ext | head -10 && cat file.ext | tail -10 ) | your_program

Answer 15

To handle pipes (streams) as well as files, add this to your .bashrc or .profile file:

headtail() { awk -v offset="$1" '{ if (NR <= offset) print; else { a[NR] = $0; delete a[NR-offset] } } END { for (i=NR-offset+1; i<=NR; i++) print a[i] }' ; }

Then you can not only

headtail 10 < file.txt

but also

a.out | headtail 10

(This still appends spurious blank lines when 10 exceeds the input's length, unlike plain old a.out | (head; tail). Thank you, previous answerers.)

Note: headtail 10, not headtail -10.

Answer 16

Building on what @Samus_ explained here about how @Aleksandra Zalcman's command works, this variation is handy when you can't quickly spot where the tail begins without counting lines.

{ head; echo "####################\n...\n####################"; tail; } < file.txt

Or if you start working with something other than 20 lines, a line count might even help.

{ head -n 18; tail -n 14; } < file.txt | cat -n

Answer 17

To print the first 10 and last 10 lines of a file, you could try this:

cat <(head -n10 file.txt) <(tail -n10 file.txt) | less

Answer 18

sed -n "1,10p; $(( $(wc -l ${aFile} | grep -oE "^[[:digit:]]+")-9 )),\$p" "${aFile}"

NOTE: The aFile variable contains the file's full path.

Answer 19

I would say that depending upon the size of the file, actively reading in its contents may not be desirable. In that circumstance, I think some simple shell scripting should suffice.

Here's how I recently handled this for a number of very large CSV files that I was analyzing:

$ for file in *.csv; do echo "### ${file}" && head ${file} && echo ... && tail ${file} && echo; done

This prints out the first 10 lines and the last 10 lines of each file, while also printing out the filename and some ellipsis before and after.

For a single large file, you could simply run the following for the same effect:

$ head somefile.csv && echo ... && tail somefile.csv

Answer 20

I did some more experimenting based mostly on suggestions here. After working a bit, I up with something very similar to another version in another comment, but with a focus more on formatting with multiple file args in addition to stdin.

This wraps nicely into a script (tentatively headtail) and uses the gnu awk. On macOs this can be installed via brew install gawk.

It can work on piped content OR a list of files as arguments. Given files, it prints a header of the file name, the head N lines, a skipped lines maker, then the tail N lines. If the head and tail overlap or would line up, it neither includes a skip marker nor does it display any duplicate lines.

#!/bin/bash
headtail_awk() {
  N=10
  gawk -v "n=${N}" -- '\
  FNR == 1 && FILENAME != "-" {
    printf "\033[036m==> %s <==\033[0m\n", FILENAME;
  }
  # print head lines
  FNR <= n { print }
  # store lines in a circular buffer
  { a[FNR % n]=$0 }
  # print non-overlapping tail lines from circular buffer.
  ENDFILE {
    if ( FNR > 2 * n ) {
      printf "\033[0;36m>>> %s lines skipped <<<\033[0m\n", FNR - 2 * n
    }
    for (i=FNR-n+1;i<=FNR;i++) {
      if ( i > n) {
        print a[i % n]
      }
    }
  }
' "$@"
}
headtail_awk "$@"

I'll leave any getopts and/or enhancements of the N=10 line window as an exercise for the reader.

sample output of multiple files (with n=3):

$ headtail -n 3 /usr/share/dict/words /usr/share/dict/propernames
==> /usr/share/dict/words <==
A
a
aa
>>> 235880 lines skipped <<<
zythum
Zyzomys
Zyzzogeton
==> /usr/share/dict/propernames <==
Aaron
Adam
Adlai
>>> 1302 lines skipped <<<
Wolfgang
Woody
Yvonne

Answer 21

this worked for me: (head -100) < source.txt > target.txt

(head -100) < source.txt takes the first 100 lines from the source.txt file and then

taget.txt pushes the 100 lines into a new file called target.txt

Initially I thought something like this should work: head -100 source.txt > target.txt but it returned an empty file.

Answer 22

Why not to use sed for this task?

sed -n -e 1,+9p -e 190,+9p textfile.txt