Pass all files in directory that match criteria to a program

Question

In windows explorer when I drag and drop 100 files over a batch file, I get an error saying that the "data area passed to a system call was too small"

I generated the batch file to take 100 arguments like so, thinking it would work

MyProg.exe %1 %2 %3 %4 %5 ... %100

MyProg takes a bunch of paths and does things to them.

So now my solution is

for %%X in (*.my_ext) do (
    MyProg.exe %%X
)

But that is initializing my program again and again since I am just passing one file to it, which sort of defeats the purpose of accepting an arbitrary number of arguments and this start-up + finish is slowing things down.

Ideally, I would like to just pass all files to my program and let it run. How would I accomplish this?

EDIT:

One idea I'm going for is this one: How to concatenate strings in a Windows batch file?

My solution now looks like this. I have two batch files get_files.bat and main.bat

get_files.bat

@echo off
set myvar=myProg.exe
for /r %%i in (*.my_ext) DO call :concat "%%i"
echo %myvar%
goto :eof

:concat
set myvar=%myvar% %1
goto :eof

main.bat

call get_files.bat > out.bat
call out.bat

I first create the command I want to call, and then I call it. This allows me to pass 100's of paths that match the given crieria to my program, though it seems like at some point I reach the limit on how long the input string can be.

The ideal solution would look something like building as long of a list as possible and passing that list to the program, and then repeat until all files have been processed. The files may be searched recursively, etc.

Is it possible to just put your program in place of `MyProg %%X`? — user2033427, Feb 21 '13 at 04:13
I meant copy and paste your program in between the `for %%X in (*.my_ext) do (` and `)`. — user2033427, Feb 21 '13 at 04:19
Have you looked at http://stackoverflow.com/questions/5185030/drag-and-drop-batch-file-for-multiple-files or http://stackoverflow.com/a/1244045/1677912? — Mogsdad, Feb 21 '13 at 04:23
user2033427, it's alright I should have made it clear. Mogsdad, For the 2nd one, I have 1000's of files to process and as mentioned explorer complains when I drop that many files over a batch file. For the 1st one...it doesn't seem like what I want. I'm looking more for the very first attempt I tried, where I can just pass a list of arguments to the program (ideally I would like to just pass 1000 strings to it and only having the program called once, but I can settle with only a fraction of that at a time). — MxLDevs, Feb 21 '13 at 04:26
Instead of squishing all your file names together, you could pass them as 100 different parameters. Of course, you can't just put %100 to get them, my idea is go through the first 9, then use shift 9 times and repeat. I would post code but I don't know exactly what you want. — BDM, Feb 21 '13 at 07:13
I would like to minimize the number of times my program has to start up. Everytime `myProg.exe arg1 arg2 arg3 ... ` is called, it would have to initialize itself and do some setup, loop over the given arguments, and then exit. Imagine if it took a minute to start-up, and a second to process an argument. My understanding of the shift command is that in the case of going through 9, it is the same as passing 9 arguments at a time. I would like it to be as greedy as possible and consume as much as it can in each iteration. — MxLDevs, Feb 21 '13 at 09:03
If this is indeed *your* program, teach it to accept a file containing a list of files to process. That is, introduce a parameter, a switch with an argument, the argument being the list file. Let the program read the list and apply the processing to the files referred in it. — Andriy M, Feb 21 '13 at 15:11
I.e. in a batch file, instead of building a huge command line, put the file names into a temporary file, then pass the file to your program using that special call I was talking about in my previous comment. — Andriy M, Feb 21 '13 at 15:19
That is a good solution, and gets around the limit on the size of a single command. That would be a step above concatenating all of them together and try using that as a single call and I would only have to start up the program once. — MxLDevs, Feb 21 '13 at 15:34

Mofi · Answer 1 · 2023-05-10T17:14:48.170

The Microsoft documentation page command prompt (Cmd.exe) command-line string limitation contained the information as quoted below in a former version on Microsoft still supported older versions of Windows like Windows NT4.0, Windows 2000 and Windows XP:

On computers running Microsoft Windows XP or later, the maximum length of the string that you can use at the command prompt is 8191 characters. On computers running Microsoft Windows 2000 or Windows NT 4.0, the maximum length of the string that you can use at the command prompt is 2047 characters.

The best method to avoid this limitation on executing an application with many files is to use a list file as suggested also by Andriy M.

For example WinRAR interprets a file name starting with @ as name of a list file containing line by line the names of the files to compress/extract.

The text editor UltraEdit supports the command line option /f"name of list file" to open all files listed line by line in the specified list file.

The file manager Total Commander supports %L, %l, %F, %f, %D, %d, %WL, %WF, %UL, %UF in a command line for an application called by Total Commander. Total Commander creates in the directory for temporary files a list file with the names of the selected files written into it line by line with one of those placeholders present, before it starts the application with the name of this list file. Total Commander even supervises the started application and deletes the list file after application terminated.

I think, in your case it would be most likely best that your application deletes the list file itself after processing all files listed in this file line by line.

Pass all files in directory that match criteria to a program

1 Answers1