9

I am trying to delete the oldest file in a tree with a script in Debian.

find /home/backups -type f \( -name \*.tgz -o -name \*.gz \) -print0 | xargs -0 ls -t | tail -1 | xargs -0 rm

But I am getting an error:

rm: cannot remove `/home/backups/tree/structure/file.2011-12-08_03-01-01.sql.gz\n': No such file or directory

Any ideas what I am doing wrong (or is there an easier/better way?), I have tried to RTFM, but am lost.

Nick Craver
  • 623,446
  • 136
  • 1,297
  • 1,155
user1076412
  • 317
  • 2
  • 5
  • 12

7 Answers7

15

The ls appends a newline and the last xargs -0 says the newline is part of the file name. Run the last xargs with -d '\n' instead of -0.

BTW, due to the way xargs works, your whole pipe is a bug waiting to happen. Consider a really long file name list produced by the find, so that the xargs -0 ls runs ls multiple times with subsets of the filenames. Only the oldest of the last ls invocation will make it past the tail -1. If the oldest file is actually, say, the very first filename output by find, you are deleting a younger file.

drevicko
  • 14,382
  • 15
  • 75
  • 97
Jens
  • 69,818
  • 15
  • 125
  • 179
  • 4
    That is not enough. Without `-d '\n'`, `xargs` splits on every whitespace, which is probably not what you want. – glglgl Dec 15 '11 at 11:25
  • perfect, thanks. By the way, what is considered a "really long file name list" please? – user1076412 Dec 15 '11 at 12:03
  • @user1076412: "long" in this case means "greater than ARG_MAX", which is something your OS will set. If all file names returned by `find` together are longer than ARG_MAX then xargs will invoke it command more than once. – sorpigal Dec 15 '11 at 12:44
  • 1
    "long" means longer than the limit imposed by xargs for a single command invocation. This might be somewhere near ARG_MAX, but may also be a constant much lower, compiled into xargs. Note that xargs can not call a command with an arbitrarily long arg list; its very purpose is to split the arg list across several command invocations should the limit be exceeded. This is no problem with commands like `rm` where multiple invocations don't change the result. But in your case there **is** a change in result. – Jens Dec 15 '11 at 13:43
  • @glglgl: while adding `-d '\n'` does remove the newline `rm` still complains about being unable to find the file. I solved the issue by piping to a [script](http://stackoverflow.com/a/4794313/1352384) but still would like to know what causes this. Fully aware this might prove difficult for lack of detail. – Sven M. Jul 08 '15 at 22:26
  • @SvenM. Maybe it could be worth to do something like `xargs -d '\n' printf '*%s*\n' | cat -A` or something in order to see what is really emitted. If you don't have the exact file name between the `*`s or you have something in-between, you have spotted your problem. – glglgl Jul 09 '15 at 04:28
3

ls emits newlines as separators, so you need to replace the second xargs -0 with xargs -d '\n'. Breaks, though, if the oldest file has a newline in its name.

thiton
  • 35,651
  • 4
  • 70
  • 100
3

Any solution involving ls is absolutely wrong.

The correct way to do this is to use find to fetch the set of files, sort to order them chronologically, filter out all but the first, then rm to delete. @Ken had this mostly right, missing only a few details.

find /home/backups -type f \( -name \*.tgz -o -name \*.gz \) -printf '%T@ %p\0' |\
    sort -z -n | \
    { IFS= read -d '' file ; [ -n "$file" ] && echo rm -f "$(cut -d' ' -f2- <<<"$file")" ; }

Remove the echo above to actually perform the deletion.

The above code works even for files which have spaces, newlines or other unusual values in the file names. It will also do nothing harmful when there are no results.

If you don't care about breaking on newlines in filenames this gets a bit easier

find /home/backups -type f \( -name \*.tgz -o -name \*.gz \) -printf '%T@ %p\n' |\
    sort -n |\
    head -n 1 |\
    cut -d' ' -f2- |\
    xargs echo rm

The difference is that we can rely on head and can use cut on a pipe instead of doing anything crazy.

sorpigal
  • 25,504
  • 8
  • 57
  • 75
2

You can also use find to print out the modification time, sort, cut and xargs at will:

find /home/backups -printf "%T@ %p\n" | sort -n | head -1 | cut -d" " -f2- | xargs ls -al
Ken
  • 77,016
  • 30
  • 84
  • 101
  • This is the best approach, but I would `cut` on a delimiter rather than rely on a fixed offset. – sorpigal Dec 15 '11 at 12:08
  • you can prise my "svn status | cut -c8-" from my cold, dead hands! But yes a delimiter would be better – Ken Dec 15 '11 at 12:12
  • Your edit has introduced a new bug: it should say `-f2-` in case the file name has spaces. – sorpigal Dec 15 '11 at 15:05
0
find /home/backups -type f \( -name \*.tgz -o -name \*.gz \) -print0 | xargs -0 stat --format '%010Y:%n' | sort -n | head -n 1 | cut -d: -f2- | xargs -d '\n' rm 

from: Sort a file list by Date in Linux (Including Subdirectories)

Sumit Singh
  • 15,743
  • 6
  • 59
  • 89
Joe
  • 1
0

Edit I missed the point of ls -t there.

Might I suggest doing it much simpler, e.g.

find /home/backups \
    -type f -iregex '.*\.t?gz$' \
    -mtime +60 -exec rm {} \;

which will delete any matching file older than a specific age (60 days, in the example)


You used tail but haven't told it to look for null-delimiters.

Regardless, here is a util that you could use to return the last 0-delimited element:

#include <string>
#include <iostream>
#include <cstdio>

int main(int argc, const char *argv[])
{
    std::cin.unsetf(std::ios::skipws);
    if (!  (freopen(NULL, "wb", stdout) && freopen(NULL, "rb", stdin) ))
    {
        perror("Cannot open stdout/in in binary mode");
        return 255;
    }

    std::string previous, element;
    while (std::getline(std::cin, element, '\0'))
    {
        previous = element; 
        // if you have c++0x support, use this _instead_ for performance:
        previous = std::move(element);
    }

    std::cout << previous << '\0' << std::flush;
}

Use it as

find /home/backups -type f \( -name \*.tgz -o -name \*.gz \) -print0 | ./mytail | xargs -0 rm 
sehe
  • 374,641
  • 47
  • 450
  • 633
  • Remember that the first `xargs` swallows the null-delimiters and the `ls` delimits with newlines, so the tail is fine. – thiton Dec 15 '11 at 11:23
  • Good idea, but for this question there is missing a sort-by-time (the `ls -t` from the OP). – thiton Dec 15 '11 at 11:38
  • @thiton: shute. I often wonder why find doesn't have sorting capabilities. – sehe Dec 15 '11 at 11:39
  • @thiton: And once more adapted my answer. I'd really try to avoid the complexity in the first place – sehe Dec 15 '11 at 11:44
0

ls -tr $(find /home/backups -name '*.gz' -o -name '*.tgz')|head -1|xargs rm -f

fge
  • 119,121
  • 33
  • 254
  • 329