Why is my Batch script skipping over some folders within a for loop?

Question

I am working on a batch script that cycles recursively through a directory on a network. With each file it reads, it grabs the path and name, writes it to a text file, and then transfers it to an excel sheet. I have a few folders that I need to skip over in the iteration. Those are within the "else if"s. However, I have several additional folders that are not being iterated through.

I should mention that it is being executed via task scheduler every 1.5 hours, and during the first few executions the script worked perfectly, but then later it is skipping folders.

Excuse the mess of code, I learned Batch whilst creating this script. In order for some things to work properly, weird things had to be done. My code is below, with a dummy folder. This is my first post on here, but I've looked all over for an explanation but had no hope. I hope I didn't break any rules.

::COMPANY NAME HERE
::MY NAME HERE - AUTOMATION INTERN
::
::An excel spreadsheet is generated containing a list of all subfiles for each project
::The spreadsheet is placed inside each project folder
::This script should run at scheduled times in order to remain updated
::

@echo off
::Make working directory begin where this file is located by pushing the path onto a stack
pushd %~dp0

::Prevent echo from happening before declaration
setlocal EnableDelayedExpansion

::Recursively access each folder inside the Projects folder
for /d /r %%G in ("\BRDATA1\Projects\*") DO (
    ::Change directory to current folder using short notation (to access deep folders)
    cd %%~sG
    ::Set cf to the name of the current folder
    set cf=%%~nG
    if "!cf!" == "Archive WIP" (
        ::DO NOT REMOVE
        ::FOR SOME REASON COMMENTS ARE NEEDED IN ORDER FOR THIS TO WORK... lol
    ) 2>nul else if "!cf!" == "Archives" (
        ::DO NOT REMOVE
        ::FOR SOME REASON COMMENTS ARE NEEDED IN ORDER FOR THIS TO WORK... lol
    ) 2>nul else if "!cf!" == "Help Document" (
        ::DO NOT REMOVE
        ::FOR SOME REASON COMMENTS ARE NEEDED IN ORDER FOR THIS TO WORK... lol
    ) 2>nul else if "!cf!" == "Recovered" (
        ::DO NOT REMOVE
        ::FOR SOME REASON COMMENTS ARE NEEDED IN ORDER FOR THIS TO WORK... lol
    ) 2>nul else if "!cf!" == "Templates" (
        ::DO NOT REMOVE
        ::FOR SOME REASON COMMENTS ARE NEEDED IN ORDER FOR THIS TO WORK... lol  
    ) 2>nul else if "!cf!" == "xxxx-Customer Name" (
        ::DO NOT REMOVE
        ::FOR SOME REASON COMMENTS ARE NEEDED IN ORDER FOR THIS TO WORK... lol
    ) 2>nul else if "!cf!" == "xxxx-Project Name" (
        ::DO NOT REMOVE
        ::FOR SOME REASON COMMENTS ARE NEEDED IN ORDER FOR THIS TO WORK... lol
    ) 2>nul else (
        echo Indexing Folder !cf!
        call :search
    )
)

::Pop the stack
echo Indexing Complete
popd

::Terminate execution
exit

:search
        ::Write each filepath inside the current folder into a temporary text file
        ::The filepath property is used and therefore efficiency is independent from file size
        dir /a-d /b /s /o:gn > list.txt
        ::Write headers on the excel output file
        echo FILENAME,FILE LOCATION > index.csv  
        ::For each filepath inside the temporary text file, Remove all text before the final \
        for /f "tokens=* delims=\" %%a in (list.txt) DO (
            ::Value = full path name (long)
            set value=%%a 
            echo !value! >nul
            ::New Value = Trimmed path name and only initial project folder name uses short path name
            set newValue=!value:*\Projects\=! 
            set newValue=!newValue:*\=!
            echo !newValue! >nul
            ::Write the filename including the extension, and then its trimmed path name 
            echo %%~na%%~xa , !newValue! >> index.csv
        )
        ::Delete the temporary text file and continue to the next project folder
        del "list.txt"

Thanks!

How do you know the additional folders are not being iterated through? What fails? Curious, do you have any .csv files loaded/locked by Excel when this runs and fails? — Kory Gill, Jan 20 '16 at 22:32
This is going to be painful in cmd.exe (batch). I would strongly recommend PowerShell instead. — Bill_Stewart, Jan 20 '16 at 22:37
Do never use wrong-label-like comments `::` within `for` loops or code blocks `()`; they might lead to unexpected behaviour; use `rem` instead... and what are the `2>nul` redirections in your `if`/`else` compound for?? — aschipfl, Jan 20 '16 at 23:22
I would reverse the `if` logic: `if /I not "!cf!"=="abc" (if /I not "!cf!"=="xyz!" ( ... (call :search)))` (the `/I` switch makes the comparison case-insensitive, which I recommend for file/dir. paths because Windows does not care about the case either) — aschipfl, Jan 20 '16 at 23:29
@aschipfl I disagree with the advice to *never* use `::` for comments. As long as you're aware of where you can put them (on their own line, and not inside a code block), I think the readability improvement over `rem` is worth it. — Ryan Bemrose, Jan 21 '16 at 00:49
@Brett General advice for debugging large batch files:Don't suppress your best debug information. Remove `@echo off` and redirect the console output to a file. Nine times out of ten, you can find the line that is broken by looking at that file. — Ryan Bemrose, Jan 21 '16 at 00:52
@RyanBemrose, the OP uses `::` within code blocks, which is not recommended -- see [this external link](http://ss64.com/nt/rem.html)... — aschipfl, Jan 21 '16 at 09:30
@Bill_Stewart I have been asked to use batch, but I can see if powershell would be acceptable. — Brett Comardelle, Jan 21 '16 at 14:34
@RyanBemrose Thanks for that advice. I've had a hard time debugging this batch file for some time now. — Brett Comardelle, Jan 21 '16 at 14:35

RLH · Accepted Answer · 2016-01-20T23:34:09.100

It's harder to diagnose the logic flow with all the nested If/else conditions. I would suggest starting with this change to see if it resolves the issue.

Edit: corrected per comment about goto. tracking condition in a var instead.

::COMPANY NAME HERE
::MY NAME HERE - AUTOMATION INTERN
::
::An excel spreadsheet is generated containing a list of all subfiles for each project
::The spreadsheet is placed inside each project folder
::This script should run at scheduled times in order to remain updated
::

@echo off
::Make working directory begin where this file is located by pushing the path onto a stack
pushd %~dp0

::Prevent echo from happening before declaration
setlocal EnableDelayedExpansion
set skip=0

::Recursively access each folder inside the Projects folder
for /d /r %%G in ("\BRDATA1\Projects\*") DO (
    ::Change directory to current folder using short notation (to access deep folders)
    cd %%~sG
    ::Set cf to the name of the current folder
    set cf=%%~nG
    if "!cf!" == "Archive WIP" set skip=1
    if "!cf!" == "Archives" set skip=1
    if "!cf!" == "Help Document" set skip=1
    if "!cf!" == "Recovered" set skip=1
    if "!cf!" == "Templates" set skip=1
    if "!cf!" == "xxxx-Customer Name" set skip=1
    if "!cf!" == "xxxx-Project Name" set skip=1
    if "!skip!" <> "1" (
        echo Indexing Folder !cf!
        call :search
    )
    set skip=0
)

::Pop the stack
echo Indexing Complete
popd

::Terminate execution
exit

:search
        ::Write each filepath inside the current folder into a temporary text file
        ::The filepath property is used and therefore efficiency is independent from file size
        dir /a-d /b /s /o:gn > list.txt
        ::Write headers on the excel output file
        echo FILENAME,FILE LOCATION > index.csv  
        ::For each filepath inside the temporary text file, Remove all text before the final \
        for /f "tokens=* delims=\" %%a in (list.txt) DO (
            ::Value = full path name (long)
            set value=%%a 
            echo !value! >nul
            ::New Value = Trimmed path name and only initial project folder name uses short path name
            set newValue=!value:*\Projects\=! 
            set newValue=!newValue:*\=!
            echo !newValue! >nul
            ::Write the filename including the extension, and then its trimmed path name 
            echo %%~na%%~xa , !newValue! >> index.csv
        )
        ::Delete the temporary text file and continue to the next project folder
        del "list.txt"

This won't work for sure! you cannot jump with `goto` to a label inside of a `for` loop or a code block `()`, because the loop/block context is going to be lost... — aschipfl, Jan 20 '16 at 23:26
Ok, that makes sense, back to my original was to use a var to track logic. — RLH, Jan 20 '16 at 23:31
You can also move the entire bode of the `for` loop into a subroutine and just `call` it; so `goto` and the label `:skip` are moved outside of a code block... — aschipfl, Jan 20 '16 at 23:34
Attempting to alter the OP code as little as possible just to let them diagnose it more easily. — RLH, Jan 20 '16 at 23:37
I will give this a try, it will take a while before it is actually tested on the server, so my response may take a while. Thanks for your help. — Brett Comardelle, Jan 21 '16 at 14:14
@RLH This has seemed to work. I have no idea why comments would cause something so odd. Perhaps it was coincidence, but it is working now. Thanks for the help! — Brett Comardelle, Jan 22 '16 at 19:01

score 0 · Answer 2 · edited May 23 '17 at 12:22

Your code is so complicated that is certainly hard to search for errors on it, but a point that have proven to cause problems is to insert double-colon "comments" inside code blocks; you should always use rem command to insert comments. Below there is an equivalent code that perform the same process than your code, but much simpler:

@echo off
::Make working directory begin where this file is located by pushing the path onto a stack
pushd "%~dp0"

::Prevent echo from happening before declaration
setlocal EnableDelayedExpansion

::Define the list of folders to omit, enclosed by slashes
set "omit=/Archive WIP/Archives/Help Document/Recovered/Templates/xxxx-Customer Name/xxxx-Project Name/"

::Recursively access each folder inside the Projects folder
for /d /r "\BRDATA1\Projects" %%G in (*) DO (
    rem Change directory to current folder using short notation (to access deep folders^)
    cd "%%~sG"
    rem If current folder is not in the list of folders to omit
    if "!omit:/%%~nG/=!" equ "%omit%" (
        echo Indexing Folder %%~nG
        call :search
    )
)

::Pop the stack
echo Indexing Complete
popd

::Terminate execution
exit


:search

::Write each filepath inside the current folder into a temporary text file
::The filepath property is used and therefore efficiency is independent from file size
dir /a-d /b /s /o:gn > list.txt

(
rem Write headers on the excel output file
echo FILENAME,FILE LOCATION

rem For each filepath inside the temporary text file, Remove all text before the final \
for /f "tokens=* delims=\" %%a in (list.txt) DO (
   rem Value = full path name (long)
   set "value=%%a"
   echo !value! >nul

   rem New Value = Trimmed path name and only initial project folder name uses short path name
   set "newValue=!value:*\Projects\=!"
   set "newValue=!newValue:*\=!"
   echo !newValue! >nul

   rem Write the filename including the extension, and then its trimmed path name 
   echo %%~NXa , !newValue!
)
) > index.csv

::Delete the temporary text file and continue to the next project folder
del "list.txt"

exit /B

Why is my Batch script skipping over some folders within a for loop?

2 Answers2