Windows Batch file - strip leading characters

Question

I have a batch file which copies some local files up to a google storage area using the gsutil tool. The gsutil tool produces a nice log file showing the details of the files that were uploaded and if it was OK or not.

Source,Destination,Start,End,Md5,UploadId,Source Size,Bytes Transferred,Result,Description
file://C:\TEMP\file_1.xlsx,gs://app1/backups/file_1.xlsx,2018-12-04T15:25:48.428000Z,2018-12-04T15:25:48.804000Z,CPHHZfdlt6AePAPz6JO2KQ==,,18753,18753,OK,
file://C:\TEMP\file_2.xlsx,gs://app1/backups/file_2.xlsx,2018-12-04T15:25:48.428000Z,2018-12-04T15:25:48.813000Z,aTKCOQSPVwDycM9+NGO28Q==,,18753,18753,OK,

What I would like to do is to

• check the status result in column 8 (OK or FAIL)
• If the status is OK then move the source file to another folder (so that it is not uploaded again).

The problem is that the source filename is appended with "file://" which I can't seem to remove, example

file://C:\TEMP\file_1.xlsx

needs to be changed into this

C:\TEMP\file_1.xlsx

I am using a for /f loop and I am not sure if the manipulation of the variables %%A is different within a for /f loop.

@echo off

rem copy the gsutil log file into a temp file and remove the header row using the 'more' command.
more +1 raw_results.log > .\upload_results.log

rem get the source file name (column 1) and the upload result (OK) from column 8
for /f "tokens=1,8 delims=," %%A in (.\upload_results.log) do (
        echo The source file is %%A , the upload status was %%B 

        set line=%%A
        set line=!line:file://:=! >> output2.txt echo !line!
        echo !line!

)

The output is like this.

The source file is file://C:\TEMP\file_1.xlsx , the upload status was OK
The source file is file://C:\TEMP\file_2.xlsx , the upload status was OK

I'm expecting it to dump the altered values out into a new file but it is not producing anything at the moment. Normally I would extract from a specific character to the end of the string with something like this but it doesn't work with my For/f loop.

%var:~7%

Any pointers or a different way of doing it greatly appreciated.

Answer 1

Since the part to remove seems fixed it is easier to use substrings.

Also using for /f "skip=1" evades he neccessity of the external command more +1 and another intermediate file.

@echo off & setlocal EnableDelayedExpansion
type NUL>output2.txt
for /f "skip=1 eol=| tokens=1,8 delims=," %%A in (.\upload_results.log) do (
    echo The source file is %%A , the upload status was %%B
    set "line=%%A"
    set "line=!line:~7!"
    echo(!line!>>output2.txt
    echo(!line!
)

Answer 2

File names and paths can contain also one or more exclamation marks. The line set line=%%A is parsed by Windows command processor a second time before execution with enabled delayed expansion. See How does the Windows Command Interpreter (CMD.EXE) parse scripts? Every ! inside the string assigned to loop variable A is on this line interpreted as begin or end of a delayed expanded environment variable reference. So the string of loop variable A is assigned to environment variable line with an unwanted modification if file path/name contains one or more exclamation marks.

For that reason it is best to avoid usage of delayed expansion. The fastest solution is for this task using a second FOR to get file:// removed from string assigned to loop variable A.

@echo off
del output2.txt 2>nul
for /F "skip=1 tokens=1,8 delims=," %%A in (upload_results.log) do (
    echo The source file is %%A , the upload status was %%B.
    for /F "tokens=1* delims=/" %%C in ("%%~A") do echo %%D>>output2.txt
)

Even faster would be without the first echo command line inside the loop:

@echo off
(for /F "skip=1 delims=," %%A in (upload_results.log) do (
    for /F "tokens=1* delims=/" %%B in ("%%~A") do echo %%C
))>output2.txt

The second solution can be written also as single command line:

@(for /F "skip=1 delims=," %%A in (upload_results.log) do @for /F "tokens=1* delims=/" %%B in ("%%~A") do @echo %%C)>output2.txt

All solutions do following:

The outer FOR processes ANSI (fixed one byte per character) or UTF-8 (one to four bytes per character) encoded text file upload_results.log line by line with skipping the first line and ignoring always empty lines and lines starting with a semicolon which do not occur here.

The line is split up on every occurrence of one or more commas into substrings (tokens) with assigning first comma delimited string to specified loop variable A. The first solution additionally assigns eighth comma delimited string to next loop variable B according to ASCII table.

The inner FOR processes the string assigned to loop variable A with using / as string delimiter to get assigned to specified loop variable file: and to next loop variable according to ASCII table the rest of the string after first sequence of forward slashes which is the full qualified file name.

The full qualified file name is output with command echo and appended either directly to file output2.txt (first solution) or first to a memory buffer which is finally at once written into file output2.txt overwriting a perhaps already existing file with that file name in current directory.

For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.

• del /?
• echo /?
• for /?

See also the Microsoft article about Using command redirection operators for an explanation of the redirections >, >> and 2>nul