Struggling to perform string substitution within a nested for loop

Question

In a Windows cmd shell batch script, I am trying to craft a mechanism to iterate over a series of directories which all have their name in the form of x.Year-Month-Day, where x is a sequential value, from 0 up to a defined limit. I want to move/rename the directories, such that the last one is deleted, and the rest increment their index by one each time the script is run. The script doesn't know what the date values will be in the individual directory names.

To wit:

@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION

REM Set work variables:
SET source=C:\Program Files\AppName\WorkDir
SET localDest=%source%\Data Backups
SET maxBackups=5

REM Get to work... 

REM Deduct 1 from maxBackups b/c of 0-based count:
SET /A limit=maxBackups-1

REM Remove any folders indexed at the limit...
FOR /D %%f IN ("%localDest%\%limit%.*") DO rd /S /Q "%%f"    

REM Iterate through the backup folders...
FOR /L %%i IN (%limit%,-1,1) DO (
    SET /A c=%%i-1
    FOR /D %%d IN ("%localDest%\!c!.*") DO (
        SET src=%localDest%\!c!.
        SET dst=%localDest%\%%i.
        REM *** Can't get this assignment to work! ***
        SET file=%%%d:!src!=!dst!%%
        MOVE /Y "%%d" "!file!"
    )
)

REM Now copy the current data folder to a new '0' index backup folder...
MD "%localDest%\0.%date%"
ROBOCOPY "%source%\Data" "%localDest%\0.%date%" *.* /S /E /COPYALL /ZB /R:5 /W:60 /XJ /NP /V %1 %2 %3 %4 %5

:End

As the comment indicates, I've tried innumerable permutations of the string-substitution, trying to create a new name for the directory with the modified index, but I cannot get the processor to do the substitution!

I've tried:

SET file=%%d:!src!=!dst!

Which produces just the three strings with the applicable punctuation, (and adding a trailing % doesn't do anything).

Prepending another % ...

SET file=%%%d:!src!=!dst!%

...produces a string that starts with a %, but only contains the values from !src! and !dst! separated by an equals sign, (=).

And, once again, adding another % to the end of the string, merely adds one to the end of the output.

I've tried innumerable combinations of % on the front and end of the assignment, but never seem to be able to get the substitution to work. I can't help but wonder if the colon from the path, (C:\), is messing with the parser. But surely a cmd shell batch processor, would be written with the understanding that paths including colons, would be exceedingly common things for such scripts to encounter, no?

I've been reading and re-reading articles here and around the web talking about how to do these things, but all the examples seem to be one-dimensional, (i.e. 'How to use delayed parsing', 'How to use variables inside loops', 'How to do string substitution' etc.). I haven't found anything demonstrating/trying to parse a string, (which contains a colon!), in real time, with value-substitution, from within a nested loop is discussed/explained/demonstrated. Trying to make it work has, thus far, proved entirely futile, and I'm at the point now of not knowing what to do/try next. What am I missing/not getting correct?

`I can't help but wonder if the colon from the Path ("C:") is messing with the parser, ...` yes, it is. `... but surely a CMD shell batch processor would be written with the understanding that Paths including colons would be exceedingly common things for such scripts to encounter, no?` No (sad but true). — Stephan, May 29 '23 at 06:19
`%%d` is a FOR _replaceable parameter_ on which the Batch _string sustitution_ doesn't works. A string sustitution works over a _variable_. In your code you should use: `set d=%%d` (convert the replaceable parameter into a variable) followed by `call set file=%%d:!src!=!dst!%%` or `for /F "tokens=1,2" %%x in ("!src! !dst!") do set file=!d:%%x=%%y!` as described with detail at [Arrays, linked lists and other data structures in cmd.exe batch script](https://stackoverflow.com/questions/10166386/arrays-linked-lists-and-other-data-structures-in-cmd-exe-batch-script/10167990#10167990) — Aacini, May 29 '23 at 06:41
@Aacini ```"%%d is a FOR *replaceable parameter* on which the Batch *string sustitution* doesn't works..."``` -- thank-you! **This** was what I didn't know/understand/realise! Along with ```pushd/popd``` I was able to get my script working (detailed below)! — NetXpert, May 30 '23 at 00:09
@Stephan -- definitely sad, but, thankfully, by using ```pushd/popd``` to stash the path, I was able to completely remove this frustrating stumbling block from the problem by just using ```.\``` instead! — NetXpert, May 30 '23 at 00:12

Stephan · Answer 1 · 2023-05-29T14:33:39.463

1

I would choose a different approach: iterating over the backup folders (having their parent folder as workingfolder simplifies things, so pushd there before). Having your nice format <number><dot><somestring> enables you to separate the number with the %%~n... modifier and have <dot><somestring> available as %%~x...

pushd %localdest%
for /f %%A in ('dir /ad /o-n /b ?.*') do (
   set /a c=%%~nA-1
   if !c! lss 0 goto :done
   ECHO move "%%A" "!c!%%~xA"
)
:done
popd

Remove the ECHO keyword when the output satisfies you to actually enable the move command.

edited May 29 '23 at 14:33

answered May 29 '23 at 07:05

Stephan

53,940
10
58
91

no, `for` doesn't parse files here, but with `/f` it parses the output of a command. `dir`'s `/ad` switch takes care of listing only folders, its `/o-n` switch lists them from highest to lowest and `/b` outputs only the names without additional information. – Stephan May 29 '23 at 14:35
I gave this a try, but for some reason it always skipped the last folder... ‍♀️ Still, thanks a ton for mentioning ```pushd``` -- that's an awsome feature! I ended up solving the problem thanks to the ```%%~n``` thing leading me to using a variable-substring function! – NetXpert May 29 '23 at 23:02

score 0 · Answer 2 · answered May 29 '23 at 08:24

@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION 
rem The following setting for the directory is a name
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.

SET "sourcedir=u:\your files\t w o"
SET /a maxfound=-1
SET /a maxindexminus1=17
:: remove variables starting #
FOR  /F "delims==" %%e In ('set # 2^>Nul') DO SET "%%e="

FOR /f "tokens=1*delims=." %%b IN ('dir /b /ad "%sourcedir%\*" ') DO (
 IF %%b gtr %maxindexminus1% (ECHO RD /s /q "%%b.%%c"
 ) ELSE (
 IF %%b gtr !maxfound! SET /a maxfound=%%b
 SET /a nextindex=%%b+1
 SET "#%%b=ren "%sourcedir%\%%b.%%c" "!nextindex!.%%c""
 )
)

FOR /L %%e IN (%maxfound%,-1,0) DO ECHO !#%%e!

GOTO :EOF

I've left the final create of 0.date out of the solution since the problem is how to change the index.

Noting that index.date and index+1.date may both exist (multiple backups made on the same day)

Read each directoryname in basic form (name only) into %%b,%%c, if %%b exceeds the maximum index, remove that directory (RD is echoed only for testing). Otherwise, retain the maximum index found in maxfound and record the required rename command in #%%b

Then loop through the rename commands in reverse, so that index+1.date is processed before index.date hence we cannot have an attempt to duplicate a name, even if the date part is the same (Again, ren is echoed only for testing)

Wow! So much here flew right over my head! I don't even know how to read most of it! It seems like it might be a lot more than what I was looking for? I managed to forge a sol'n by taking the one suggestion to cast the ```FOR``` replacement var ```%%d``` to a standalone variable, then using substring features to create the new target folder name and leveraging ```pushd``` ... — NetXpert, May 29 '23 at 23:11

NetXpert · Accepted Answer · 2023-05-30T00:19:26.607

Thanks to all of the suggestions, comments and examples, I finally managed to cobble together a solution to my issue!

It really came down to not realising/understanding that FOR substitution parameters didn't function like 'normal' environment variables wrt the string manipulation features. Also, learning about pushd/popd was pretty awesome too!

Here's a pared-down version of what I came up with:

@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION

REM Set work variables:
SET source=C:\Program Files\AppName\WorkDir
SET localDest=%source%\Data Backups
REM NOTE: 'maxBackups' CANNOT be greater than 10!
SET maxBackups=5

REM Get to work... 

REM Deduct 1 from maxBackups b/c of 0-based count:
SET /A limit=maxBackups-1
PUSHD %localDest%

REM Remove any/all folders indexed at the limit...
FOR /D %%f IN (".\%limit%.*") DO rd /S /Q "%%f"    

REM Iterate through the existing backup folders...
FOR /L %%i IN (%limit%,-1,1) DO (
    SET /A c=%%i-1
    FOR /D %%d IN (".\!c!.*") DO (
        SET folderSuffix=%%d
        MOVE /Y "%%d" ".\%%i!folderSuffix:~3!"
    )
)

REM Now create a new '0' index backup folder...
MD ".\0.%date%"

REM ...and copy the current data folder's contents to it!
ROBOCOPY "%source%\Data" ".\0.%date%" *.* /S /E /COPYALL /ZB /R:5 /W:60 /XJ /NP /V /J /TEE /COMPRESS %1 %2 %3 %4 %5

:End
POPD

..and yes, I'm aware that things will go tragically sideways if maxBackups exceeds 10, but each folder is nearly 20GB in size currently (and growing) and I don't genuinely forsee us ever keeping many more than the handful (i.e. ~5) that we're currently retaining. I just felt that using the variable made implementing tweaks/changes easier down the road if we ever decided we wanted to add/remove a few (i.e. going with 7 to have daily images, or down to 2 or 3 to save space, or something similar ‍♀️).

Struggling to perform string substitution within a nested for loop

3 Answers3