Add information to a bufferWriter before flushing it java

Question

I understand that the BufferedWriter stores the information before I write in in a file before executing the act of writing it in a file using flush(), Append() etc..

I gather information from multiple sources, so currently what I am doing is looping in each source and appending it directly to the file each time, but what I'm trying to accomplish is to add all the information in the BufferedWriter and after finishing the loop, writing it to the file, how could that be done?

I am trying to improve performance by not flushing the data into the file so many times. The performance is issue because this might loop 1 million times.

Here is what I'm currently doing:

1. Open BufferedWriter
2. read data from a different source and storing in the buffer
3. appending stored data in a text file(here the buffer is emptied)
4. repeating steps 2.- and 3.- 50 times
5. closing text file

Here is what I'm trying to do:

1. Open BufferedWriter
2. read data from a different source and storing in the buffer
3. repeat step 2.- 50 times
4. append all the data collected(the data gathered over the 50 loops)
5. close the file

here is the code.

 for (int mainLoop = 0; mainLoop < 50; mainLoop++){

        try {
            BufferedWriter writer = writer = new BufferedWriter(new 
FileWriter
                    ("path to file in computer" + mainLoop + ".txt", true));


        for(int forloop = 0; forloop < 50; forloop++) {
            final Document pageHtml= 
Jsoup.connect("link to a page").get();


            Elements body = pageHtml.select("p");

            writer.append(System.getProperty("line.separator"));
            writer.append(System.getProperty("line.separator"));
            writer.append(body.text());

            System.out.println(forloop);
        }
        writer.close();
        } catch (IOException e) {
            e.printStackTrace();
        }continue;
    }

Answer 1

i am trying to improve performance by not flushing the data into de file so many times

Are you flushing the data manually after each write? Don't do that.

Otherwise, specify a larger size when you instantiate your BufferedWriter.

You have the option of using a StringBuilder to aggregate the output first. However, I assume you have more output than you want to store in memory.

Finally, is there really a performance cost?

===

The BufferedWriter will optimize the actual writes it performs. As long as you specify a large buffer size, e.g., 10,000, multiple small writes to the buffer will not cause a write until the buffer is full. I see a comment that you are "clearing" the buffer. Don't do that. Leave the BufferedWriter alone and let it do its thing.

If you are accumulating information and then, for some reason, discarding it, use a StringBuilder to accumulate and then write the StringBuild content to the Writer.

Answer 2

A buffered writer will flush when you instruct it to. And also any time the buffer becomes full. It can be tricky to determine when buffer becomes full. And really, you should not care. A buffered writer will improve performance regardless of precisely when it flushes. Instead, your output code should use BufferedWriter just like any other Writer.

I also see in your code that you repeatedly open and close the output file. You almost certainly don't need to do that. Instead open and close the file at a higher level in your program, so it remains open for each iteration.