Remove CRLF from large text file

Question

I have a 1Gig file with couple of million lines. Trying to remove CRLF at the end of each line. Have tried EmEdit, GVIM, Notepad++, UltraEdit. Only EMEdit will load file, but it is unbelievably slow on the replace. Maybe 10 lines a second. Don;t have enough years left in my life to wait for this to complete.

I am not a programmer so I am strictly using the UI dialog boxes. Don't really know what else to do. Thanks

http://stackoverflow.com/questions/6951687/find-and-replace-text-in-a-47gb-large-file — Ali Sheikhpour, Apr 18 '15 at 20:16
For anyone looking for a solution for a file small enough to load in Notepad ++, see https://superuser.com/questions/545461/replace-carriage-return-and-line-feed-in-notepad/546031 — Chris Halcrow, Nov 27 '19 at 05:11

score 0 · Answer 1 · answered Apr 18 '15 at 20:25

Textpad will open 1gb files and I've done what you say with it.. But it will take a REALLY long time.

I don't know about how it compares in speed to the other editors you mentioned for str-replace.

Anyway, here are instructions to do it from textpad if you're interested: Since you're restricted to dialogs and windows, my answer is restricted to install textpad 7, open it, push F8 key to open the search/replace dialog.

Search for \r\n, replace with nothing. And check the regex box or it will take you literally. Depending on the file you may be looking for \r, or \n, or both. Then click 'replace next' a few times to make sure it works, finally click replace all if satisfied with the first few.

Hope it helps.

score 0 · Answer 2 · answered Apr 21 '15 at 06:15

UltraEdit can open any file of any size, even files with more than 4 GB on a machine having just 2 GB RAM. Such large files should be opened in UltraEdit without usage of a temporary file (modifications are permanent and cannot be undone) and with some options configured different to default for better performance on working on large files, see power tip Large file text editor.

However, for this replace it is not necessary to open the large file at all in UltraEdit.

Just start UltraEdit and open Replace in Files from menu Search.

Enter as search string ^p and as replace string nothing.
Files listed is by default selected.
Use button ... right of Directory edit field to select the directory containing the large file.
Enter in edit field In files/types the name of the file or a string with wildcard(s) which matches just the large file in the selected directory.
Make sure that Regular expressions is not selected as ^p means DOS/Windows line termination (carriage return + line-feed) for non regular expression or UltraEdit regular expression replaces only.
Click on button Replace all.

Alternatively with having Regular expressions checked and Unix or Perl selected, the search string [\r\n]+ finds any carriage return and any line-feed in the file and replaces it with nothing. With UltraEdit regular expression the search string for such a replace would be [^r^n]+.

score -1 · Answer 3 · answered Apr 18 '15 at 20:20

-1

in notepad++, have you tried the following:

Set search mode to \r\n
In the Find what text box, enter \r\n
Leave the Replace with field empty
Click Replace All

Or is that what you have tried already?

answered Apr 18 '15 at 20:20

Chris

598
8
23

The OP states that Notepad++ has been tried and won't load the file as it's too large. There's already a solution here https://superuser.com/questions/545461/replace-carriage-return-and-line-feed-in-notepad/546031, for replacing CRLF using Notepad ++. – Chris Halcrow Nov 27 '19 at 05:13

Remove CRLF from large text file

3 Answers3