Delete certain range of lines in a txt file containing keyword via a batch file

Question

I'm trying to delete certain lines that contains a keyword in a txt file using CMD or VBS.

I've read this but it's not the same. I want to delete a range of lines.

Original Text File:

ABCDEFGXXX
ABCD
A
AE
AXXXLKGUSP
0000ASD
ASD

Processed Text File:

0000ASD
ASD

I want to delete the range of lines containing the first instance of 'XXX' and the second.There are only two instances of this 'XXX'. The number of lines between the two instances are random and there may be a scenario where the two instances are in the same line. The 4 zeroes also appear after the line of the 2nd instance of 'XXX'. Please note that it may contain characters below, so it may choke if you try to process.

---------------------EDIT 08/04/2015 7:41 PM----------------------------------- "XXX" is all caps... and the text file may contain characters below.. it shows a single line in notepad though. IDK what they are

PK
    ¨‘G_t¥0  8ˆ     XXXç¸[~-ÄWÀ¨Ì’gÝ

Answer 1

Assuming you've mentioned VBS to indicate it's acceptable to have a non-pure-batch-syntax solution, here's a powershell one-liner which you can call from a .bat.

It reads the text file in default system encoding, other useful values are UTF8 and Unicode.
A 100MB file is processed in 2 seconds.

@echo off
set "string=XXX"
set "infile=input file.txt"
set "outfile=output file.txt"
set "encoding=default"

powershell -ExecutionPolicy bypass -c "$txt=(get-content '%infile%' -raw -encoding default); $i=$txt.indexof('%string%'); if($i -ge 0) { $j=$txt.indexof('%string%',$i+'%string%'.length); if($j -ge 0) {$k=$txt.indexof(\"`n\",$j); if($k -ge $j){$txt2=$txt.substring($k)} else {$txt2=''} $txt.substring(0,[math]::max(0,$txt.lastindexof(\"`n\",$i))) + $txt2 | out-file '%outfile%' -encoding %encoding%}}"
pause

• -ExecutionPolicy bypass is added to allow execution of powershell on a non-admin user account.
• PowerShell 3.0 and newer is required, it comes by default with Windows 7 SP1, 8, 10.

Answer 2

This is really simple with JREPL.BAT - a pure script based utility (hybrid JScript/batch) that runs natively on any Windows machine from XP onward. The key feature is it supports multi-line regular expression search and replace.

The following overwrites the original file:

jrepl "^.*?XXX[\s\S]*XXX.*\n?" "" /m /f "test.txt" /o -

If you want to create a new file, then simply specify a file name instead of - for the /O option:

jrepl "^.*?XXX[\s\S]*XXX.*\n?" "" /m /f "input.txt" /o "output.txt"

Or you could omit the /O option entirely and print the result to the screen (stdout)

jrepl "^.*?XXX[\s\S]*XXX.*\n?" "" /m /f "input.txt"

Use call jrepl ... if you put the command within a batch script.

EDIT: Here is a pure batch solution. Normally I've abandoned using batch for text processing because a robust solution requires too much insanity. The code below is fairly robust and optimized, but even so, still has the following limitations:

• Lines limited to <8k
• Searched keyword cannot begin with *, and cannot contain =, !, or "
• The search ignores case, with no option to make it case sensitive

There may be some more I have missed.

But the code does preserve empty lines, and does not choke on ! in the content. (handling these possibilities is the cause for much of the complexity)

@echo off
setlocal disableDelayedExpansion

set "in=input.txt"
set "out=output.txt"
set "find=XXX"

set "cnt=0"
>"%out%" (
  for /f "delims=" %%A in ('findstr /n "^" "%in%"') do (
    set "ln=%%A"
    setlocal enableDelayedExpansion
    set "ln=!ln:*:=!"
    if defined ln if !cnt! equ 0 (
      set "test=!ln:*%find%=!"
      if !test! neq !ln! set "cnt=1"
    ) else set "test=!ln!"
    if !cnt! neq 1 echo(!ln!
    if defined test if !cnt! neq 2 if "!test:%find%=!" neq "!test!" set "cnt=2"
    for %%N in (!cnt!) do endlocal&set "cnt=%%N"
  )
)

Answer 3

This is a pure CMD implementation:

@echo off

rem DEFINITIONS:
set "KEYWD=XXX"
set "INFILE=original.txt"
set "OUTFILE=modified.txt"

setlocal EnableExtensions EnableDelayedExpansion

rem GET_LINE_NUMBERS:
set "NUMONE="
set "NUMTWO="
for /F %%F in ('findstr /N /L /C:"%KEYWD%" "%INFILE%"') do (
  for /F "delims=:" %%N in ("%%F") do (
    if not defined NUMONE (
      set "NUMONE=%%N"
    ) else (
      set "NUMTWO=%%N"
    )
  )
)
if not defined NUMTWO set "NUMTWO=%NUMONE%"

rem RETURN_BEFORE_BLOCK:
set /A "COUNT=0"
rem.> "%OUTFILE%"
for /F "delims=" %%L in ('findstr /N /R "^" "%INFILE%"') do (
  set /A "COUNT+=1"
  if !COUNT! geq !NUMONE! (
    goto :NEXT
  ) else (
    setlocal DisableDelayedExpansion
    set "LINE=%%L"
    setlocal EnableDelayedExpansion
    echo(!LINE:*:=!
    endlocal
    endlocal
  ) >> "%OUTFILE%"
)

rem RETURN_AFTER_BLOCK:
:NEXT
if defined NUMTWO set "SKIP=skip=!NUMTWO!"
for /F "%SKIP% delims=" %%L in ('findstr /N /R "^" "%INFILE%"') do (
  setlocal DisableDelayedExpansion
  set "LINE=%%L"
  setlocal EnableDelayedExpansion
  echo(!LINE:*:=!
  endlocal
  endlocal
) >> "%OUTFILE%"

endlocal

The code consists of four sections (see the remarks rem):

• DEFINITIONS: here you need to define the search keyword, the input and the output files;
• GET_LINE_NUMBERS: this section searches for the line numbers of the two occurrences of the given keyword; the resulting line numbers are stored in the respective variables NUMONE and NUMTWO; if only one line with (a) match(es) is found, NUMTWO is set to NUMONE; if no match is found, both variables remain empty;
• RETURN_BEFORE_BLOCK: here everything up to but not including the line with the first keyword match is output; it relies on the fact that goto breaks any ongoing for loop context;
• RETURN_AFTER_BLOCK: in this section every line after the second keyword match is returned and appended to the output of the previous section; here the for /F option argument skip is built dynamically;

Answer 4

Here is another hybrid JScript / Batch solution.

Assume the text file is in data.txt we can invoke a replacer script:

cscript //nologo j.js .*XXX.*\n[\s\S]*?.*\n*XXX.* "" < data.txt

Where j.js is a supporting script written in Microsoft JScript:

var txt = WScript.StdIn.ReadAll();
var pattern = new RegExp( WScript.Arguments.Item(0), "g" );
var newvalue = WScript.Arguments.Item(1);
txt = txt.replace( pattern, newvalue );
WScript.StdOut.Write( txt );

A golf-optimized version of the j.js script is:

WScript.StdOut.Write(WScript.StdIn.ReadAll().replace(new RegExp(WScript.Arguments.Item(0),"g"),WScript.Arguments.Item(1)));

Generally speaking, the j.js usage is:

cscript //nologo j.js pattern_regular_expression new_value < input.dat > output.dat

Where input.dat refers to an input text/data file. If < input.dat is omitted, input will be taken from the user input. And output.dat refers to an output text/data file. If > output.dat is omitted, the output will be shown on the console.