39

I have recently been copying and moving a large number of files (~400,000). I know that there are limitations on the number of arguments that can be expanded on the Bash command line, so I have been using xargs to limit the numbers produced.

Out of curiosity, I wondered what the maximum number of arguments that I could use was, and I found this post saying that it was system-dependant, and that I could run this command to find out:

$ getconf ARG_MAX

To my surprise, the anwser I got back was:

2621440

Just over 2.6 million. As I said, the number of files that I am manipulating is much less than this -- around 400k. I definitely need to use the xargs method of moving and copying these files, because I tried using a normal mv * ... or cp * ... and got a 'Argument list too long' error.

So, do the mv and cp commands have their own fixed limit on the number of arguments that I can use (I couldn't find anything in their man pages), or am I missing something?

Lee Netherton
  • 21,347
  • 12
  • 68
  • 102
  • Why do you have to specify each file? Can't you just specify the directory they're in or something like that? – Christoffer Hammarström Nov 15 '10 at 14:02
  • 1
    @Christoffer I'm not just copying a directory from one place to another, I'm copying specific files, renaming bunches of files, moving certain files from one place to another. I have no problem in actually performing these operations. I was just curious as to what the specific limitations of the `cp` and `mv` commands were. – Lee Netherton Nov 15 '10 at 14:14

2 Answers2

34

As Ignacio said, ARG_MAX is the maximum length of the buffer of arguments passed to exec(), not the maximum number of files (this page has a very in-depth explanation). Specifically, it lists fs/exec.c as checking the following condition:

PAGE_SIZE*MAX_ARG_PAGES-sizeof(void *) / sizeof(void *)

And, it seems, you have some additional limitations:

On a 32-bit Linux, this is ARGMAX/4-1 (32767). This becomes relevant if the average length of arguments is smaller than 4. Since Linux 2.6.23, this function tests if the number exceeds MAX_ARG_STRINGS in <linux/binfmts.h> (2^32-1 = 4294967296-1). And as additional limit, one argument must not be longer than MAX_ARG_STRLEN (131072).

Wyatt Anderson
  • 9,612
  • 1
  • 22
  • 25
8

ARG_MAX is the maximum length of the arguments to the exec(3) functions. A shell is not required to support passing this length of arguments from its command line.

Ignacio Vazquez-Abrams
  • 776,304
  • 153
  • 1,341
  • 1,358
  • But bash does not enforce _any_ limit, says [this](http://www.in-ulm.de/~mascheck/various/argmax/); it just tries the `exec` and passes along an OS error if the argument list is too long. – alexis Jan 20 '14 at 12:54