0

I have two file's with 3 GB each to do a comparison and store the difference data into third file.

Following is the sample data.

File 1: June_01_2018.csv

enter image description here

File 2: June_02_2018.csv

enter image description here

Note: I want to store the difference into the third file with the column headers.

Expected Result: The third file should contain the records from ID 11 to 15 with the column headers.

My try: fc command in command prompt,

>fc June_01_2018.csv June_02_2018.csv > June_Diff.csv

But getting unexpected result:

enter image description here

MAK
  • 6,824
  • 25
  • 74
  • 131
  • What about this: `< "June_01_2018.csv" set HEAD=""` and `> "June_Diff.csv" echo(%HEAD%` and `>>"June_Diff.csv" findstr /V /X /L /G:"June_01_2018.csv" "June_02_2018.csv"`; this should work if the second file contains all data of the first one, like shown in your example... – aschipfl Jun 05 '18 at 22:25
  • @aschipfl, I tried with `findstr` but getting error `FINDSTR: Cannot read strings from file`. – MAK Jun 06 '18 at 05:15
  • Strange... what encoding do you use -- ASCII/ANSI? how long are the lines? perhaps you are facing a limitation of `findstr` (too long lines, or too many lines) -- consult this: [What are the undocumented features and limitations of the Windows FINDSTR command?](https://stackoverflow.com/q/8844868) – aschipfl Jun 06 '18 at 10:15

1 Answers1

0

Try copying the header record first, then append the fc output.

powershell -NoProfile -Command ^
    "Get-Content -Path 'June_01_2018.csv' | Select-Object -First 1" >June_Diff.csv
fc June_01_2018.csv June_02_2018.csv >> June_Diff.csv
lit
  • 14,456
  • 10
  • 65
  • 119
  • Total records in the first file are 10 millions and second file 11 millions. So there be 1 million more or less records getting appended in latest file. I tried with ‘fc file1.txt file2.txt > output.txt’ but getting error ‘Failed to resych’. – MAK Jun 05 '18 at 12:28
  • This sounds like a problem of the `fc` command and not one of getting the header record into the output. The Microsoft answer. https://msdn.microsoft.com/en-us/library/ms827336.aspx Do you have the option of installing another tool? – lit Jun 05 '18 at 12:30
  • Yes I have option. – MAK Jun 05 '18 at 12:54
  • A first step might be to install (for free) Cygwin or MinGW and use `diff`, but that might not be the format you like. There is a free `WinDiff`, but I do not know if it can be scripted. If you can purchase a tool, `BeyondCompare` is very strong and has some scripting capability. https://www.scootersoftware.com/ – lit Jun 05 '18 at 14:03
  • Okay will try this. – MAK Jun 05 '18 at 14:16
  • [KDiff3](http://kdiff3.sourceforge.net/) is a free (GPL) excellent graphical compare (and merge) tool. 10 million lines might not give the best responsiveness, but give it a try. – hlovdal Jun 05 '18 at 15:35