Gathering list of file extensions - "The command line is too long" workaround

Question

I am currently using this batch file to scan through a Windows file system and save a .txt document of all the file extensions in that system:

Cmd Line Command:

    NameOfBatchFile.bat >List.txt

BatchFile Code:

    @echo off

    set target=%1
    if "%target%"=="" set target=%cd%

    setlocal EnableDelayedExpansion

    set LF=^


    rem Previous two lines deliberately left blank for LF to work.

    for /f "tokens=*" %%i in ('dir /b /s /a:-d "\\PathOfMyWindowsDirectory"') do (
        set ext=%%~xi
        if "!ext!"=="" set ext=FileWithNoExtension
        echo !extlist! | find "!ext!" > nul
        if not !ERRORLEVEL! == 0 set extlist=!extlist!!ext!:
    )

    echo %extlist::=!LF!%

    endlocal

The code works great on small folders but if I provide it a folder with too many subfolders, the command line will process then provide the following error:

    The command line is too long.
    The command line is too long.
    The command line is too long.
    The command line is too long.
    The command line is too long.
    The command line is too long.
    The command line is too long.
    The input line is too long.

I can't edit the filesystem to decrease subfolders, does anyone know another way to get this to work?

You look to be using a UNC path in the DIR command - are you checking a machine over a network? It can make a difference to the solutions. — foxidrive, Jul 21 '14 at 17:14

MC ND · Accepted Answer · 2014-07-21T16:51:59.420

3

The problem in your code is the concatenation of elements inside a variable, that can generate a long list of extensions that will end generating a excesively long command line.

@echo off
    setlocal enableextensions disabledelayedexpansion

    set "target=%~1"
    if "%target%"=="" set "target=%cd%"

    for /r "%target%" %%a in (*) do if not defined "\%%~xa\" (
        if "%%~xa"=="" (echo(FileWithNoExtension) else (echo(%%~xa)
        set ""\%%~xa\"=1"
    )

    endlocal

This uses the environment to store the information of seen extensions by setting a variable for each one. If the variable is not set, this is the first time the extension is found and is echoed to console.

edited Jul 21 '14 at 16:51

answered Jul 21 '14 at 16:46

MC ND

69,615
8
84
126

That looks good. You don't need delayed expansion if you check for the variable being defined. I wonder what it will do on a path too long? – foxidrive Jul 21 '14 at 16:55
@foxidrive, no delayed expansion is used to prevent problems with exclamations in file extensions. And the problem in the OP code is not the path length, but the concatenation of the extensions. – MC ND Jul 21 '14 at 17:01
@foxidrive, have you tested if the `for` command is able to retrieve the extension (the requirement in this question) of the files inside that folder? – MC ND Jul 21 '14 at 17:20
It works fine. My file manager must have been whingeing at me but there is no problem. – foxidrive Jul 21 '14 at 17:34
This worked well - it took some time to process, but as it was running over 37,000 files, that was to be expected. For posterity's sake, the command used to run this file also required the file name in the command NameOfBatchFile.bat \\PathOfMyWindowsDirectory >List.txt – user3730104 Jul 21 '14 at 19:21
My answer gets an `extra 4 extensions` compared to yours and Aacini's solution as `for /r` ignores hidden files. – foxidrive Jul 21 '14 at 23:36

score 1 · Answer 2 · answered Jul 21 '14 at 22:26

I think this is the fastest way to get the result to this problem.

@echo off
setlocal

set target=%1
if "%target%"=="" set target=%cd%

for /R "%target%" %%a in (*.*) do set ext[%%~Xa]=1

for /F "tokens=2 delims=[]" %%a in ('set ext[') do (
   if "%%a" equ "=1" (
      echo FileWithNoExtension
   ) else (
      echo %%a
   )
)

Previous method may be easily modified in order to get the number of files that have each extension; just modify the set ext[%%~Xa]=1 by set /A ext[%%~Xa]+=1 and modify the tokens in the for /F accordingly.

Aacini - try my answer which uses repl.bat and your uniq.bat (with a modification to lowercase) and is very swift. I get an `extra 4 extensions` compared to yours and MC ND's solution as `for /r` ignores hidden files. — foxidrive, Jul 21 '14 at 23:35

foxidrive · Answer 3 · 2014-07-21T23:38:35.257

This gives you a sorted list of all the extensions very quickly.

@echo off

set "target=%~1"
if "%target%"=="" set target=%cd%

dir "%target%" /b /s /a-d |repl ".*(\..*)" "$1" |repl ".*\\.*" "FileWithNoExtension"|sort|uniq >file.txt

This uses a helper batch file called repl.bat (by dbenham) - download from: https://www.dropbox.com/s/qidqwztmetbvklt/repl.bat

This uses a helper batch file called uniq.bat (by aacini) - download from: https://www.dropbox.com/s/71o38a4ljnqgqjh/uniq.bat

Place repl.bat and uniq.bat in the same folder as the batch file or in a folder that is on the path.

Gathering list of file extensions - "The command line is too long" workaround

3 Answers3

Linked