87

I have a generated txt file. This file has certain lines that are superfluous, and need to be removed. Each line that requires removal has one of two string in the line; "ERROR" or "REFERENCE". These tokens may appear anywhere in the line. I would like to delete these lines, while retaining all other lines.

So, if the txt file looks like this:

Good Line of data
bad line of C:\Directory\ERROR\myFile.dll
Another good line of data
bad line: REFERENCE 
Good line

I would like the file to end up like this:

Good Line of data
Another good line of data
Good line

TIA.

paxdiablo
  • 854,327
  • 234
  • 1,573
  • 1,953
chrome
  • 1,101
  • 2
  • 9
  • 12

5 Answers5

123

Use the following:

type file.txt | findstr /v ERROR | findstr /v REFERENCE

This has the advantage of using standard tools in the Windows OS, rather than having to find and install sed/awk/perl and such.

See the following transcript for it in operation:

C:\>type file.txt
Good Line of data
bad line of C:\Directory\ERROR\myFile.dll
Another good line of data
bad line: REFERENCE
Good line

C:\>type file.txt | findstr /v ERROR | findstr /v REFERENCE
Good Line of data
Another good line of data
Good line
paxdiablo
  • 854,327
  • 234
  • 1,573
  • 1,953
  • 1
    Great, just the line I was looking for. – chrome Jan 07 '09 at 15:37
  • 6
    Had this exact problem and found it here on SO. God bless Stack Overflow!! – Matt Rogish May 08 '09 at 19:55
  • 27
    To be complete: you need to add "> output.txt" (without quotes) to have the output written in a file. – Burkhard Sep 14 '09 at 18:37
  • Thanks, this was quick and easy just the way I like it! – Kuberchaun Aug 03 '10 at 15:10
  • @StarShip3000, I'd be _very_ surprised if piping a text file through `findstr` would _modify_ the lines in it somehow. It should just remove whole lines. You might want to open a question if that's the case, I'll keep an eye open. – paxdiablo Aug 04 '10 at 13:56
  • @paxdiablo I just tried to reproduce with a manually created test case and couldn't get it to reproduce. I know this happened, but maybe this command against my data wasn't the cause. I'll consider myself wrong until I can prove otherwise. – Kuberchaun Aug 04 '10 at 15:31
  • 3
    thanks, this helped me out but a word of caution, I had some issues with phrases including spaces and had to use /c: so the command would look like this: type file.txt | findstr /v /c:"ERROR" | findstr /v /c:"REFERENCE". Also if you need to strip lines off that you know are at the top fo the file, pipe it through more + where is the number of lines to strip off. – Apeiron Nov 12 '11 at 00:21
  • Good trick. But beware, there is an issue with too long lines: "FINDSTR: Line 1 is too long." – Roland Pihlakas Jan 04 '15 at 17:03
  • Beware that similarly to the `findstr` command there is also a `find` command that does the same, and also fails on long lines. BUT it fails silently. In some forums they suggest that it is able to work on longer lines. On my computer it was not able to do that and even failed without any warning. – Roland Pihlakas Jan 04 '15 at 17:51
  • @paxdiablo, Is there anyway to crop away contents within a line? For example if we want the bad lines displayed too, but without the characters `ERROR` and `REFERENCE`. (basically a cut operation). – Pacerier Aug 25 '15 at 13:13
  • @Pacerier, that would be better asked as a separate question since a/ you'll get a better range of responses than just mine, irrespective of the fact mine would be awesome beyond belief :-) and b/ that's the way SO works, comments are second class citizens, the questions and answers are the "meat and veg" of the site. I suppose I should have said comments were the sauce/ketchup but I *do* love a mixed metaphor :-) – paxdiablo Aug 26 '15 at 05:29
  • while you're here, can you please tell how to remove new lines or empty lines? thanks. – raja777m Jun 28 '17 at 22:01
  • @raja777m, this isn't meant to be a conversation board. If you want an answer to a question, you need to, well, *ask a question!* I especially love the comment "while you're here" given my last comment on this question was about a year ago :-) – paxdiablo Jun 29 '17 at 00:57
  • @paxdiablo; You came back, because of the notification right ;) Anyway, I already had find in my "type" query, so, didn't wanted add anything else. I found the solution: FINDSTR /V /R /C:"^$" Removes empty lines. One more reason to ask here and not a new thread is, I used the whole string from this answer, and other users don't need to google search for the lines right anymore. – raja777m Jun 29 '17 at 18:07
  • 2
    why is it deleting everything in my file test.java if I write `type test.java | findstr /v "ECHO is on." > test.java` but it do what is expected if I write it to a new file like `type test.java | findstr /v "ECHO is on." > test2.java` – Mohammed Julfikar Ali Mahbub Feb 28 '19 at 18:14
  • @Mohammad: again, that's a *question* in its own right and should be asked as such. However, in this case, the answer is simple. The redirection (creation/overwrite) to the file is done by the shell, which truncates the file *before* your `type` reads it. That means all your first command sees is an empty file because it's been cleared out. If you redirect to a *different* file (as per your second command), then *that* file (rather than your original file) is created/overwritten. The original file remains untouched. – paxdiablo Mar 01 '19 at 00:39
  • I tried this type D:/e5/datastream/package.json | findstr /v eikon-framework | findstr /v df-core but it doesn't delete the lines – kittu Dec 18 '19 at 06:55
  • @kittu, the demo in this answer is an actual transcript so it does work. Are you saying that the command you gave is outputting lines containing (for example) "df-core"? If so, you should ask a separate question (referencing this one if you wish). By making a comment to this answer, you notify just me, With a question, the great hordes of SO will see it. Just make sure you provide more details (the file content, the output, etc). – paxdiablo Dec 18 '19 at 07:00
  • I tried findstr /V "eikon-framework df-core" D:\e5\datastream\package.json > D:\e5\datastream\package.json but it deletes all the contents if I try replace the data in the same file and If I use different out put file name then it works. How do I update the same file? – kittu Dec 18 '19 at 07:17
  • @kittu, the output redirection starts before the process is even run, meaning your file is gone before you start reading it. You can do something like: `move package.json backup.file ; findstr /v "eikon..." backup.file >package.json`. That way there's no empty-overwrite and you have the original backed up in case something goes wrong. – paxdiablo Dec 18 '19 at 07:37
  • @paxdiablo This: move D:\e5\datastream\package.json D:\e5\datastream\backup.json ; findstr /V "eikon-framework df-core" D:\e5\datastream\backup.json > D:\e5\datastream\package.json still overwrites with empty content. – kittu Dec 18 '19 at 08:49
  • @kittu, sorry, that should have been `move /y package.json backup.json & indstr /v "eikon..." backup.json >package.json`. The one I gave was a syntax error. Better yet is to use `&&` so that the `findstr` only runs if the `move` succeeds. I also added `/y` to the `move` command so it overwrites an existing backup. See https://stackoverflow.com/questions/8055371/how-do-i-run-two-commands-in-one-line-in-windows-cmd for detail. – paxdiablo Dec 18 '19 at 10:21
  • @paxdiablo Thank you so much. Adding /y overwrites the existing file – kittu Dec 18 '19 at 11:30
58

You can accomplish the same solution as @paxdiablo's using just findstr by itself. There's no need to pipe multiple commands together:

findstr /V "ERROR REFERENCE" infile.txt > outfile.txt

Details of how this works:

  • /v finds lines that don't match the search string (same switch @paxdiablo uses)
  • if the search string is in quotes, it performs an OR search, using each word (separator is a space)
  • findstr can take an input file, you don't need to feed it the text using the "type" command
  • "> outfile.txt" will send the results to the file outfile.txt instead printing them to your console. (Note that it will overwrite the file if it exists. Use ">> outfile.txt" instead if you want to append.)
  • You might also consider adding the /i switch to do a case-insensitive match.
Alex from Jitbit
  • 53,710
  • 19
  • 160
  • 149
Rick
  • 4,575
  • 1
  • 26
  • 20
  • type !INSTALLED_APPS! | findstr /v "@ //" > temp_filter_out_junk.txt >nul 2>&1 – Mike Q Aug 20 '13 at 21:32
  • I tried this `type D:/e5/datastream/package.json | findstr /v eikon-framework | findstr /v df-core` but it doesn't delete the lines – kittu Dec 18 '19 at 06:55
  • I tried findstr /V "eikon-framework df-core" D:\e5\datastream\package.json > D:\e5\datastream\package.json but it deletes all the contents if I try replace the data in the same file and If I use different out put file name then it works. How do I update the same file? – kittu Dec 18 '19 at 07:17
  • 1
    @kittu Use ">>" instead of ">". From the answer: Use ">> outfile.txt" instead if you want to append. You can't alter the same file in that single command. – Rick Dec 18 '19 at 17:53
  • 1
    Well since the question is about replacing in a file, how do you alter the current file? Everything I tried just leads to the file being empty – Tofandel Jan 19 '22 at 12:15
7

If you have sed:

sed -e '/REFERENCE/d' -e '/ERROR/d' [FILENAME]

Where FILENAME is the name of the text file with the good & bad lines

prosoitos
  • 6,679
  • 5
  • 27
  • 41
Rob
  • 3,687
  • 2
  • 32
  • 40
1

If you have perl installed, then perl -i -n -e"print unless m{(ERROR|REFERENCE)}" should do the trick.

mirod
  • 15,923
  • 3
  • 45
  • 65
0

It seems that using the FIND instead of the FINDSTR can support also unicode characters. e.g. type file.txt | find /v "Ω"

Hary Dee
  • 21
  • 7