How to replace text in file too large for a String

Question

How can I replace text in a file that is too large for a String? I have been using the following to replace text in most of the files I've seen:

File f = new File("test.xml");
String content = FileUtils.readFileToString(f, "UTF-8");

content = content.replaceFirst("some text", "new text");
FileUtils.writeStringToFile(f, content, "UTF-8");

This works well for files that are normal sized. However, some of the files I have been getting are very large (too large to store in a String) and they cause an overflow. How can I replace text in those files?

You could try and use a buffered `Reader`, `Writer` and replace within each line while writing back to another file. — Mena, Oct 25 '16 at 13:46
BufferedReader and BufferedWriter are the way to read/write file in Java in most of the case. — Cédric O., Oct 25 '16 at 13:50
This answer details how to read files more efficiently: http://stackoverflow.com/questions/326390/how-do-i-create-a-java-string-from-the-contents-of-a-file?rq=1 — reden, Oct 25 '16 at 13:57
You might be able to use a random access file, https://docs.oracle.com/javase/7/docs/api/java/io/RandomAccessFile.html — patrik, Oct 25 '16 at 14:21

score 3 · Answer 1 · answered Oct 25 '16 at 14:41

3

You can use sed command to replace string of large file like below [Linux]

String[] cmdarr = {"bash", "-c", "sed 's/oldstring/newstring/g' input.txt > output.txt"};
Process runCmd = Runtime.getRuntime().exec(cmdarr);

in Windows you can install sed, use some thing like

String[] cmdarr = {"cmd", "/c", "sed 's/oldstring/newstring/g' input.txt > output.txt"};
Process runCmd = Runtime.getRuntime().exec(cmdarr);

more on Link

answered Oct 25 '16 at 14:41

Md Ayub Ali Sarker

10,795
4
24
19

I tried to run the last element of "cmdarr" on command line and it worked very fast. – Main Pal Oct 04 '17 at 04:25

score 1 · Answer 2 · answered Oct 25 '16 at 13:52

1

Try using a FileInputStream to read it line-wise

FileInputStream fs = new FileInputStream("test.xml");
BufferedReader reader = new BufferedReader(new InputStreamReader(fs));

etc.

Sorry if there are any errors in the code, i can't try the code myself right now

answered Oct 25 '16 at 13:52

DLIK

50
7

replace in the while and write it in a newly generated file :) – DLIK Oct 25 '16 at 13:56

score 1 · Accepted Answer · answered Oct 25 '16 at 15:05

try {
            File f=new File("test.xml");
            BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(f)));
            String content=null;

            while((content=reader.readLine())!=null)
            {
                content = content.replaceFirst("some text", "new text");
                System.out.println(content);
            }

        } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

what if we got a single lined file (e.g., minified XML)? – Mohammad Faisal Dec 01 '21 at 06:55 — Mohammad Faisal, Dec 01 '21 at 06:55

score -1 · Answer 4 · answered Oct 25 '16 at 15:08

-1

Try to use FART(find and replace text) for windows: fart -c big_filename.txt "find_this_text" "replace_to_this"

answered Oct 25 '16 at 15:08

Sergey Frolov

1,317
1
16
30

How to replace text in file too large for a String

4 Answers4