Bash script to delete keep last N files only

Question

I have a bunch of log files and I would like to keep only the last 3 (N) files, the most recent ones. How do you do this in bash elegantly?

I have this script but it's kind of long.

The files could be something like:

my-file-pattern.log.2019-10-01
my-file-pattern.log.2019-10-02
my-file-pattern.log.2019-10-03
and so on

My script:

#!/bin/bash
function keepLastNOnly(){
    local WORKDIR="$1"
    local PATTERN="$2"
    local MAX=$3
    cd $WORKDIR 
    COUNT=$(ls -t $WORKDIR | grep $PATTERN|wc -l|grep -o -E '[0-9]+')
    while [ $COUNT -gt $MAX ]; do
        local TODEL=$(ls -t $WORKDIR | grep $PATTERN |tail -n 1)
        rm -rf "$TODEL"
        COUNT=$(ls -t $WORKDIR | grep $PATTERN|wc -l|grep -o -E '[0-9]+')
    done
}
keepLastNOnly "/MyDirectory/" "my-file-pattern.log" 3

Any shorter way?

Can you use [`find`](https://stackoverflow.com/questions/13868821/shell-script-to-delete-directories-older-than-n-days)? — Sean Bright, Oct 18 '19 at 13:28
If you are willing to switch to `zsh`, it's just `rm my-file-pattern.log.*([-3,-1])`. There are other options available to modify how the expansion is sorted. This has the additional benefit of being safe for *all* valid file names, including those that contain newlines. — chepner, Oct 18 '19 at 13:31

score 3 · Accepted Answer · answered Oct 18 '19 at 13:36

3

Keeping to the style of the question:

#!/bin/bash

function keepLastNOnly() {
    local WORKDIR="$1"
    local PATTERN="$2"
    local MAX=$3

    for file in `ls -t1 $WORKDIR | grep $PATTERN | head -n -3`; do
       rm $file
    done
}

The clue is the -n -3 flag to head.

Note that this version will keep the latest modified files, not the latest according to file name.

answered Oct 18 '19 at 13:36

Eivind Tagseth

71
3

1

for file in `ls -v $WORKDIR | grep -E "$PATTERN" | head -n -${MAX}`; do – RonPringadi Mar 05 '20 at 02:46

score 2 · Answer 2 · answered Oct 18 '19 at 13:23

2

consider using logrotate which does all that for you, you just need to configure it:

https://linux.die.net/man/8/logrotate

answered Oct 18 '19 at 13:23

Chris Maes

35,025
12
111
136

score 0 · Answer 3 · answered May 22 '23 at 14:05

0

Sort by time, new in a top. Skip 10 files in path, perform deletion of rest.

for rmfile in $(ls -lt /path/to/files/* | tail -n +11 | awk '{print $9}'); do
  echo "Deleting file ${rmfile}"
  rm -f ${rmfile}
done

answered May 22 '23 at 14:05

Nik

1

score 0 · Answer 4 · answered May 22 '23 at 23:08

#!/bin/bash

function keepLastNOnly(){
    local workdir="$1"
    local pattern="$2"
    local max="$3"
    awk -v max="$max" '
        NR<=max{ print "keeping " $0; next } 
        {   print "removing", $0 
            print |"rm -f " $0 
        }
    ' < <(find "${workdir}" -type f -name "${pattern}" | sort -r)
} 

keepLastNOnly . "my-file-pattern.log*" 3

$ ./test.sh 
keeping ./my-file-pattern.log.2019-10-08
keeping ./my-file-pattern.log.2019-10-07
keeping ./my-file-pattern.log.2019-10-06
removing ./my-file-pattern.log.2019-10-05
removing ./my-file-pattern.log.2019-10-04
removing ./my-file-pattern.log.2019-10-03
removing ./my-file-pattern.log.2019-10-02
removing ./my-file-pattern.log.2019-10-01

Bash script to delete keep last N files only

4 Answers4