Making filewriting process even more faster

Question

I am writing the below code that this code basically writes to a file through buffered writer, I am using JDK 1.5, now my query is that basically this code works perfectly and writes to the file , the file is big let say about the final size of the file is about 1GB, now please let me is there any other alternative offered by Java in JDK 1.5 which could make file writer process even more faster and improve the performance?

 File file = new File(FilePath + getFileName());
 try (BufferedWriter bw = new  BufferedWriter(new FileWriter(file))) {
   for (Posuser posuser : totalusers) {
     for (potook bk : books) {
       if ((posuser.getUpdatedBy() == null)
           && (posuser.getUpdateddateformat() == null)) {
         bw.write("aaa"+"\r\n");
       } else if (posuser.getUpdatedBy() == null
                  && posuser.getUpdateddateformat() != null) {
         bw.write("bbb+"\r\n");
       } else if (posuser.getUpdatedBy() != null
                  && posuser.getUpdateddateformat() == null) {
         bw.write("ccc"+"\r\n");
       } else {
         bw.write("ddd"+\r\n");
       }
     }
   }
 } catch (IOException, AnotherException e) {
   logger.error(e.getMessage());
 }

score 1 · Answer 1 · edited May 23 '17 at 11:43

1

I would check out the java.nio package. This SO question/answer details more info re. performance increases.

My experience with larger files sizes has been that java.nio is faster than java.io. Solidly faster. Like in the >250% range.

edited May 23 '17 at 11:43

Community

1
1

answered Aug 16 '13 at 14:56

Brian Agnew

268,207
37
334
440

could you please let me know is nio is there in jdk 1.5 – Ram Kumar Aug 16 '13 at 14:57
According to [one study here](http://java.dzone.com/articles/file-copy-java-%E2%80%93-benchmark) the advantage of nio starts mostly disappears once you get to huge files like OP is talking about. @RamKumar - You need to use nio differently in Java 1.5 (no java.nio.files package), but the essentials are there. Take a look at `java.nio.channels`. You'll have to hunt around for some older tutorials on the web that show how to use nio in Java 1.5. – Ted Hopp Aug 16 '13 at 15:08
Well, i thought `NIO` is usable only for files copying. The advantage should be no need to read whole file to memory and then to write it to disk, but just copy it via buffer between two IN/OUT channels. Well, in this case all structures all in memory and they need to be written to disk, so i did not see any improvement. – Milan Baran Aug 16 '13 at 15:15

score 1 · Answer 2 · answered Aug 16 '13 at 15:08

This should be at least memory efficient. Blog: String intern The string + is costly operation, you should use StringBuilder instead.

   final String aaaString = "aaa".intern();
    final String bbbString = "bbb".intern();
    final String cccString = "ccc".intern();
    final String dddString = "ddd".intern();
    final String EOL = "\r\n".intern();

    File file = new File(FilePath + getFileName());
    try (BufferedWriter bw = new  BufferedWriter(new FileWriter(file))) {
      for (Posuser posuser : totalusers) {
        for (potook bk : books) {
          if ((posuser.getUpdatedBy() == null)
              && (posuser.getUpdateddateformat() == null)) {
            bw.write(aaaString);
            bw.write(EOL);
          } else if (posuser.getUpdatedBy() == null
                     && posuser.getUpdateddateformat() != null) {
            bw.write(bbbString);
            bw.write(EOL);
          } else if (posuser.getUpdatedBy() != null
                     && posuser.getUpdateddateformat() == null) {
            bw.write(cccString);
            bw.write(EOL);
          } else {
            bw.write(dddString);
            bw.write(EOL);
          }
        }
      }
    } catch (IOException, AnotherException e) {
      logger.error(e.getMessage());
    }

also please suggest if I had (aaaString+bbbString) then how could I make this more memory efficient — Ram Kumar, Aug 16 '13 at 15:21
Do not join them just write one by one. `bw.write(aaaString); bw.write(bbbString);` — Milan Baran, Aug 16 '13 at 15:24
@RamKumar - For a single expression, joining with a `StringBuilder` is no more efficient than using string concatenation with `+`. You gain efficiency when you can reuse a `StringBuilder` in place of multiple concatenations in separate statements. But the best approach is to avoid concatenation when it is not needed, as Milan suggests. — Ted Hopp, Aug 16 '13 at 16:02

score 0 · Answer 3 · answered Aug 16 '13 at 15:04

0

It's not enough just to use a BufferedWriter, you need to tweak the buffer size (second constructor parameter). The larger the buffer, the most performance you'll get. But remember, if you set the buffer size to 1MB, you'll need to be sure that reserving 1MB of RAM to every concurrent call of this code isn't an issue. Also, Brian's answer (java.nio package) is very nice. Combine the two: change to java.nio AND tweak the buffer size until you find the magic number to your use case.

answered Aug 16 '13 at 15:04

samuelgrigolato

192
1
6

It might also boost performance to buffer at the stream level as well, rather than using a `FileWriter`. – Ted Hopp Aug 16 '13 at 15:10
@TedHopp Please also adivise what will be the appropriate buffer size that I can enter into the constructor for writing a file of 1 gb – Ram Kumar Aug 16 '13 at 15:22
@RamKumar - It's hard to say for sure; this is best determined by benchmarking and it will vary with your execution environment and the load on your system. The best buffer size is probably some multiple of the disk block size. I believe that the default buffer size for a `BufferedWriter` is 8192 (8KB). – Ted Hopp Aug 16 '13 at 15:59

Making filewriting process even more faster

3 Answers3