Findstr and REGEX: how to define that it could be a lot of additional characters between the lines?

Question

I'm new to regular expressions in batch for windows. I have the following log file:

14:35:32 Destination File: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SME99.ISO

14:34:43 Operation failed! - Duration: 00:01:05
14:37:02 Average Read Rate: 3.573 KiB/s (20.7x) - Maximum Read Rate: 4.812 KiB/s (27.9x)

I want to check whith a help of findstr and regex if "Operation failed!" occurs between the words .ISO and "Average Read Rate". Between all these three lines could occur additional lines from the LOG. That's what I tried to do :

@echo off

findstr /R /N /C:"Operation Failed!" ImgBurn.log

pause

Edit 1: Actually the entire log file looks like this:

; //****************************************\\
;   ImgBurn Version 2.5.8.0 - Log
;   Mittwoch, 14 Juni 2017, 19:30:05
; \\****************************************//
;
;
I 14:30:49 ImgBurn Version 2.5.8.0 started!
I 14:30:49 Microsoft Windows 8 Professional x64 Edition (6.2, Build 9200)
I 14:30:49 Total Physical Memory: 16.680.588 KiB  -  Available: 8.279.396 KiB
I 14:30:49 Initialising BS_Robots...
I 14:30:49 BS_SDK Version 2.2.0.277  Build 2013.02.22
I 14:30:49 Initialising SPTI...
I 14:30:49 Searching for Auto Loader devices...
I 14:31:07 -> Auto Loader 1 - Info: Nimbie NB21 1.13.11.26
I 14:31:07 Found 1 Auto Loader!
I 14:31:07 Searching for SCSI / ATAPI devices...
I 14:31:07 -> Drive 1 - Info: ASUS BW-16D1HT 1.01 (E:) (USB 2.0)
I 14:31:08 -> Drive 2 - Info: PLDS DVD-ROM DH-16D7S WD11 (D:) (SATA)
I 14:31:08 Found 1 DVD-ROM and 1 BD-RE XL!
I 14:33:22 Operation Started!
I 14:33:22 Source Device: [0:0:0] ASUS BW-16D1HT 1.01 (E:) (USB)
I 14:33:22 Source Media Type: CD-ROM
I 14:33:22 Source Media Supported Read Speeds: 4x; 8x; 10x; 16x; 24x; 32x; 40x; 48x
I 14:33:22 Source Media Supported Write Speeds: 16x
I 14:33:22 Source Media Sectors: 15.735
I 14:33:22 Source Media Size: 32.225.280 bytes
I 14:33:22 Source Media Volume Identifier: DGZfP BB66
I 14:33:22 Source Media File System(s): ISO9660; Joliet
I 14:33:22 Read Speed (Data/Audio): MAX / 40x
I 14:33:22 Destination File: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\DGZfP BB66.iso
I 14:33:22 Destination Free Space: 338.360.074.240 Bytes (330.429.760,00 KiB) (322.685,31 MiB) (315,12 GiB)
I 14:33:22 Destination File System: NTFS
I 14:33:23 File Splitting: Auto
I 14:33:34 Read Speed - Effective: 4x
I 14:33:35 Reading Session 1 of 1... (1 Track, LBA: 0 - 15736)
I 14:33:35 Reading Track 1 of 1... (MODE1/2048, LBA: 0 - 15736)
I 14:34:43 Image MD5: 40835c9c32af1b954fbd1e8eb878ab3c
I 14:34:43 Exporting Graph Data...
I 14:34:43 Graph Data File: C:\Users\NekhayenkoO\AppData\Roaming\ImgBurn\Graph Data Files\ASUS_BW-16D1HT_1.01_MITTWOCH-14-JUNI-2017_14-33_N-A.ibg
I 14:34:43 Export Successfully Completed!
I 14:34:43 Operation Successfully Completed! - Duration: 00:01:05
I 14:34:43 Average Read Rate: 484 KiB/s (2.8x) - Maximum Read Rate: 639 KiB/s (3.7x)
I 14:35:32 Operation Started!
I 14:35:32 Source Device: [0:0:0] ASUS BW-16D1HT 1.01 (E:) (USB)
I 14:35:32 Source Media Type: CD-ROM
I 14:35:32 Source Media Supported Read Speeds: 4x; 8x; 10x; 16x; 24x; 32x; 40x; 48x
I 14:35:32 Source Media Supported Write Speeds: 48x
I 14:35:32 Source Media Sectors: 128.648
I 14:35:32 Source Media Size: 263.471.104 bytes
I 14:35:32 Source Media Volume Identifier: SME99
I 14:35:32 Source Media File System(s): ISO9660
I 14:35:32 Read Speed (Data/Audio): MAX / 40x
I 14:35:32 Destination File: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SME99.ISO
I 14:35:32 Destination Free Space: 338.327.040.000 Bytes (330.397.500,00 KiB) (322.653,81 MiB) (315,09 GiB)
I 14:35:32 Destination File System: NTFS
I 14:35:32 File Splitting: Auto
I 14:35:34 Read Speed - Effective: 48x
I 14:35:35 Reading Session 1 of 1... (1 Track, LBA: 0 - 128647)
I 14:35:35 Reading Track 1 of 1... (MODE1/2048, LBA: 0 - 128647)
I 14:37:02 Image MD5: 8bb58d62b8e4952dc2f029c8d50ac984
I 14:37:02 Exporting Graph Data...
I 14:37:02 Graph Data File: C:\Users\NekhayenkoO\AppData\Roaming\ImgBurn\Graph Data Files\ASUS_BW-16D1HT_1.01_MITTWOCH-14-JUNI-2017_14-35_N-A.ibg
I 14:37:02 Export Successfully Completed!
I 14:37:02 Operation Successfully Completed! - Duration: 00:01:12
I 14:37:02 Average Read Rate: 3.573 KiB/s (20.7x) - Maximum Read Rate: 4.812 KiB/s (27.9x)
I 14:37:57 Operation Started!
I 14:37:57 Source Device: [0:0:0] ASUS BW-16D1HT 1.01 (E:) (USB)
I 14:37:57 Source Media Type: DVD+R (Book Type: DVD-ROM) (Disc ID: MCC-003-00)
I 14:37:57 Source Media Supported Read Speeds: 2x; 4x; 6,3x; 8,3x; 10,3x; 12,1x
I 14:37:57 Source Media Supported Write Speeds: 4x
I 14:37:57 Source Media Sectors: 1.649.280 (Track Path: PTP)
I 14:37:57 Source Media Size: 3.377.725.440 bytes
I 14:37:57 Source Media Volume Identifier: Biomasse
I 14:37:57 Source Media Volume Set Identifier: 41628eeb        Biomasse
I 14:37:57 Source Media Application Identifier: SONIC SOLUTIONS IMAGESCRIPT
I 14:37:57 Source Media Implementation Identifier: DVD Producer 1.0
I 14:37:57 Source Media File System(s): ISO9660; UDF (1.02)
I 14:37:57 Read Speed (Data/Audio): MAX / 40x
I 14:37:57 Destination File: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\Biomasse.iso
I 14:37:57 Destination Free Space: 338.066.927.616 Bytes (330.143.484,00 KiB) (322.405,75 MiB) (314,85 GiB)
I 14:37:57 Destination File System: NTFS
I 14:37:57 File Splitting: Auto
I 14:37:59 Read Speed - Effective: 5x - 12,1x
I 14:38:02 Reading Session 1 of 1... (1 Track, LBA: 0 - 1649279)
I 14:38:02 Reading Track 1 of 1... (MODE1/2048, LBA: 0 - 1649279)
I 14:43:30 Image MD5: 92955210a762c83ebd9dff7eda538dc3
I 14:43:30 Exporting Graph Data...
I 14:43:30 Graph Data File: C:\Users\NekhayenkoO\AppData\Roaming\ImgBurn\Graph Data Files\ASUS_BW-16D1HT_1.01_MITTWOCH-14-JUNI-2017_14-37_MCC-003-00.ibg
I 14:43:30 Export Successfully Completed!
I 14:43:30 Operation Failed! - Duration: 00:05:15
I 14:43:30 Average Read Rate: 10.471 KiB/s (7.7x) - Maximum Read Rate: 14.410 KiB/s (10.7x)
I 14:44:19 Operation Started!
I 14:44:19 Source Device: [0:0:0] ASUS BW-16D1HT 1.01 (E:) (USB)
I 14:44:19 Source Media Type: CD-ROM
I 14:44:19 Source Media Supported Read Speeds: 4x; 8x; 10x; 16x; 24x; 32x; 40x; 48x
I 14:44:19 Source Media Supported Write Speeds: 48x
I 14:44:19 Source Media Sectors: 110.527
I 14:44:19 Source Media Size: 226.359.296 bytes
I 14:44:19 Source Media Volume Identifier: SAMPE36
I 14:44:19 Source Media Volume Set Identifier: NOT_SET
I 14:44:19 Source Media Application Identifier: TOAST ISO 9660 BUILDER COPYRIGHT (C) 1997-2002 ROXIO, INC. - HAVE A NICE DAY
I 14:44:19 Source Media File System(s): ISO9660; Joliet
I 14:44:19 Read Speed (Data/Audio): MAX / 40x
I 14:44:19 Destination File: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SAMPE36.ISO
I 14:44:19 Destination Free Space: 334.689.107.968 Bytes (326.844.832,00 KiB) (319.184,41 MiB) (311,70 GiB)
I 14:44:19 Destination File System: NTFS
I 14:44:19 File Splitting: Auto
I 14:44:21 Read Speed - Effective: 48x
I 14:44:25 Reading Session 1 of 1... (1 Track, LBA: 0 - 110526)
I 14:44:25 Reading Track 1 of 1... (MODE1/2048, LBA: 0 - 110526)
I 14:45:42 Image MD5: 960268d9b234ed2a90b8ec9800d717da
I 14:45:42 Exporting Graph Data...
I 14:45:42 Graph Data File: C:\Users\NekhayenkoO\AppData\Roaming\ImgBurn\Graph Data Files\ASUS_BW-16D1HT_1.01_MITTWOCH-14-JUNI-2017_14-44_N-A.ibg
I 14:45:42 Export Successfully Completed!
I 14:45:42 Operation Successfully Completed! - Duration: 00:01:05
I 14:45:42 Average Read Rate: 3.400 KiB/s (19.7x) - Maximum Read Rate: 4.576 KiB/s (26.6x)

Answer 1

FIND and FINDSTR are written preliminary for searching for a string in each line of a file and output the entire line containing the found string. Multi-line searches are not possible and regular expression support of FINDSTR is very limited. Run in a command prompt window find /? and findstr /? for help on those two commands.

However, a FOR loop with string substitutions and string comparisons could be used to process the log file.

Example lines of ImgBurn.log:

14:25:32 Destination File: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SME98.ISO
14:27:02 Average Read Rate: 3.573 KiB/s (20.7x) - Maximum Read Rate: 4.812 KiB/s (27.9x)

14:33:32 Destination File: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SME99.ISO
14:34:43 Operation failed! - Duration: 00:01:05
14:37:02 Average Read Rate: 3.573 KiB/s (20.7x) - Maximum Read Rate: 4.812 KiB/s (27.9x)

The wanted output for this log file:

Success on: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SME98.ISO
Error on:   C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SME99.ISO

The batch code to get this output on processing the lines of ImgBurn.log:

@echo off
if not exist "ImgBurn.log" goto :EOF

setlocal EnableExtensions EnableDelayedExpansion
set "Searching=0"
set "IsoFileName="

for /F "usebackq delims=" %%L in ("ImgBurn.log") do (
    set "Line=%%L"
    if !Searching! == 0 (
        if /I "!Line:~-4!" == ".iso" (
            set "Searching=1"
            for /F "tokens=3*" %%I in ("!Line!") do set "IsoFileName=%%J"
        )
    ) else (
        if not "!Line:Average Read Rate=!" == "!Line!" (
            echo Success on: !IsoFileName!
            set "Searching=0"
        ) else if not "!Line:Operation failed=!" == "!Line!" (
            echo Error on:   !IsoFileName!
            set "IsoFileName="
            set "Searching=0"
        )
    )
)
endlocal

The FOR command reads from specified log file line by line and processes further all non empty lines and all lines not starting with ; which is the default for eol (end of line) option.

Each line is assigned completely to loop variable L because of using option delims= which turns off default splitting up of the read line into tokens using space and horizontal tab as delimiters.

The line is assigned to environment variable Line.

Note: On the command line set "Line=%%L" a single exclamation mark in line read from file is removed by Windows command interpreter because of delayed expansion being enabled. And if there would be two exclamation marks in a line the string between would be interpreted as environment variable name being replaced with value of that environment variable name or nothing in case of no environment variable exists with a name matching case-insensitive the string between the two exclamation marks in the line. But all that should not matter here as long as the ISO file name and its path do not contain also 1 or more exclamation marks.

Next an IF condition is used to determine if currently searching for Average Read Rate and for Operation Failed is active or searching for ISO file name should be performed as on value of Searching is 0.

It is expected that the last 4 characters are case-insensitive equal .iso on searching for ISO file name when the line contains the ISO file name.

If this condition is true for current line, Searching is toggled to 1 and the ISO file name with full path is determined from the current line using one more FOR loop.

This inner FOR loop processes now the string of environment variable Line and not a file as no usebackq option used like on outer FOR loop. The line string is split up into the 4 tokens time string, Destination, File: and rest of line using the default delimiters space/tab whereby third token File: is assigned to loop variable I and rest of line to next loop variable J according to ASCII table because of the option tokens=3*.

Once an ISO file name is read from a line the searching changes.

Now the batch code substitutes case-insensitive all occurrences of Average Read Rate in string value of environment variable Line by an empty string and compares the result with unmodified line string. This string comparison is not equal if the current line contains Average Read Rate which means Searching is toggled again now to value 0 and the image burning was successful which is output accordingly.

But if the current line does not contain Average Read Rate, it is checked if the current line contains case-insensitive Operation failed. If this is the case because line with string substitution is not equal unmodified line, the image burning was not successful which is output accordingly. Then Searching toggles again to value 0 for searching in log file for next ISO file name at end of a line.

For log lines format as posted in question with Edit 1 the option string tokens=3* of inner FOR loop must be changed to tokens=4* because there is an additional space/tab delimited string at beginning of each line in comparison to initial posted log lines.

The output with this small modification for Edit 1 log file contents:

Success on: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\DGZfP BB66.iso
Success on: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SME99.ISO
Error on:   C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\Biomasse.iso
Success on: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SAMPE36.ISO

For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.

• echo /?
• endlocal /?
• for /?
• goto /?
• if /?
• set /?
• setlocal /?

Answer 2

Batch files are known for lousy regexp support.

If you would like to search just for the string without the surroundings (no regexp) it is simple:

@echo off
SET /p file=<ImgBurn.log

findstr /m "Operation failed^!" ImgBurn.log > nul
IF %errorlevel% EQU 0 (
    ECHO "FOUND^!"
) ELSE (
    ECHO "NOT FOUND^!"
)
PAUSE

If you insist on using regexp I would turn to PowerShell launched via .bat file:

The batch file would look like this:

@ECHO OFF
powershell.exe -noninteractive -NoProfile -ExecutionPolicy Bypass -Command "& {.\find_string_in_file.ps1};"
PAUSE

The powershell file (find_string_in_file.ps1) doing the regexp:

$file = Get-Content C:\prg\PowerShell\_Snippets\Inputfile\ImgBurn.log
$search_for = [regex]"(?!\.ISO)Operation failed!(?!Average Read Rate)"
$found = $search_for.Match($file) 

If ($found.Captures[0].value -eq 'Operation failed!') {
    Write-Host "FOUND!"
} Else {
    Write-Host "NOT FOUND!"
}

Edit: If you want to get multiple results you can do it the following way (after the $found = $search_for.Match($file)):

while ($found.Success) {
    $resultslist.Add($found.Value) | out-null
    $found = $found.NextMatch()
 } 

Then you just printout with foreach.

Edit2: I have preapred an example ImgBurn.log file from your comment:

14:35:32 Destination File: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SME98.ISO

14:34:43 Operation failed! - Duration: 00:01:05
14:37:02 Average Read Rate: 3.573 KiB/s (20.7x) - Maximum Read Rate: 4.812 KiB/s (27.9x)

14:35:32 Destination File: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SME99.ISO

14:34:43 Operation failed! - Duration: 00:01:05
14:37:02 Average Read Rate: 3.573 KiB/s (20.7x) - Maximum Read Rate: 4.812 KiB/s (27.9x)

14:35:32 Destination File: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SME100.ISO

14:34:43 Operation Successful - Duration: 00:01:05
14:37:02 Average Read Rate: 3.573 KiB/s (20.7x) - Maximum Read Rate: 4.812 KiB/s (27.9x)

14:35:32 Destination File: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SME101.ISO

14:34:43 Operation failed! - Duration: 00:01:05
14:37:02 Average Read Rate: 3.573 KiB/s (20.7x) - Maximum Read Rate: 4.812 KiB/s (27.9x)

14:35:32 Destination File: C:\Users\NekhayenkoO\Desktop\LOG Dateien CD Imaging\SME102.ISO

14:34:43 Operation Successful - Duration: 00:01:05
14:37:02 Average Read Rate: 3.573 KiB/s (20.7x) - Maximum Read Rate: 4.812 KiB/s (27.9x)

I have promised to put a complete solution (batch file stays the same). Here it is:

$file = Get-Content C:\prg\PowerShell\_Snippets\Inputfile\file.log
# Matching regex
$search_for = [regex]"(?!\.ISO)Operation failed!(?!Average Read Rate)"
$found = $search_for.Match($file)

# initialize ArrayList
[System.Collections.ArrayList]$list = @()
while ($found.Success) {
    # out-null is there for output to be cancled, otherwise outputs the number of ArrayList element (e.g. 0, 1, 2, etc.)
    $list.Add($found.value) | out-null
    $found = $found.NextMatch()
}

# reading the ArrayList
foreach ($found_value in $list) {
    Write-Host "Found: $found_value"
}

However, this solution is in my eyes this is not that useful.

Get failed filenames based on regex

You are probably searching for the files that failed. I have prepared a modification which shows you the failed files:

# Do not forget the -Raw switch.  Without it it will not match correctly
# Raw siwtch forces powershell to read a text file as a single line of text, 
# not as an array of strings  created by end-of-line returns.
$file = Get-Content C:\prg\PowerShell\_Snippets\Inputfile\file.log -Raw

# Matching regex
$search_for = [regex]"\w+\.ISO(?=(\r|\n)*.*Operation failed)"
$found = $search_for.Match($file)

# initialize ArrayList
[System.Collections.ArrayList]$list = @()
while ($found.Success) {
    # out-null is there for output to be cancled, otherwise outputs the number of ArrayList element (e.g. 0, 1, 2, etc.)
    $list.Add($found.value) | out-null
    $found = $found.NextMatch()
}

# reading the ArrayList
foreach ($found_value in $list) {
    Write-Host "Operation failed at file: $found_value"
}

Answer 3

The Batch file below do exactly what you requested, that is, "check with a help of findstr and regex if "Operation failed" occurs between the words .ISO and "Average Read Rate" in a log file", including additional empty lines between these three lines.

@echo off
setlocal EnableDelayedExpansion

rem Define LF variable containing a LineFeed
set LF=^
%Empty line 1/2, don't remove%
%Empty line 2/2, don't remove%

rem Define CR variable containing a CarriageReturn
for /F %%a in ('copy /Z "%~F0" nul') do set "CR=%%a"

findstr /R /C:"\.ISO[!CR!!LF!]*.*Operation failed.*[!CR!!LF!]*.*Average Read Rate" test.txt > NUL
if errorlevel 1 (echo Not found) else echo STRING FOUND

Tested on Windows 8.1