How to fix ' Argument list too long' error?

Question

ive executed this command to delete all occurrences of "#" from a huge amount of files (about 3000)in folder train,

perl -pi -e "s/#//g" /Users/Kian/Desktop/Acsegment/espslabs/train/*

but I got this error: /bin/bash: /usr/bin/perl: Argument list too long

Can anyone suggest a way to avoid this error?

mpapec · Accepted Answer · 2015-01-14T12:55:34.033

8

Leave globing to perl instead of bash which has limitations,

perl -pi -e 'BEGIN{ @ARGV = glob(pop) } s/#//g' "/Users/Kian/Desktop/Acsegment/espslabs/train/*"

or when are spaces in globed directory,

perl -MFile::Glob=bsd_glob -pi -e 'BEGIN{ @ARGV = bsd_glob(pop) } s/#//g' "/Users/Kian/Desktop/Acsegment/espslabs/train/*"

edited Jan 14 '15 at 12:55

answered Jan 14 '15 at 12:08

mpapec

50,217
8
67
127

Thank you for respond Сухой27, I tried your solution, but I receive again the same error. – KianStar Jan 14 '15 at 12:15
Both work for me. Also (in case anyone shares an initial misconception of mine) I wondered if `"files*"` was getting expanded by the shell instead of `perl` and needed to be changed to single-quotes. I was wrong though: `echo "*"` and `echo '*'` behave the same, and neither is expanded by the shell like `echo *`. (This also checks out with `print $#ARGV` in perl) – Joshua Goldberg Mar 12 '23 at 18:37

ikegami · Answer 2 · 2015-01-14T12:40:28.420

That's what xargs is all about.

ls /Users/Kian/Desktop/Acsegment/espslabs/train |
   xargs perl -i -pe's/#//g'

find can do it too, and it allows you to select which files to modify much more flexibly.

find /Users/Kian/Desktop/Acsegment/espslabs/train \
   -type f -maxdepth 1 \
   -exec perl -i -pe's/#//g' {} +

Of course, you could also generate the file list in Perl.

perl -i -pe'BEGIN { @ARGV = map glob, @ARGV } s/#//g' \
    '/Users/Kian/Desktop/Acsegment/espslabs/train/*'

But you have to escape spaces in the path unless you use bsd_glob.

perl -i -pe'
   use File::Glob qw( bsd_glob );
   BEGIN { @ARGV = map bsd_glob($_), @ARGV }
   s/#//g
' '/Users/Kian/Desktop/Acsegment/espslabs/train/*'

How to fix ' Argument list too long' error?

2 Answers2

Linked