Best way to write String to file using java nio

Question

I need to write(append) huge string to flat file using java nio. The encoding is ISO-8859-1.

Currently we are writing as shown below. Is there any better way to do the same ?

public void writeToFile(Long limit) throws IOException{
     String fileName = "/xyz/test.txt";
     File file = new File(fileName);        
     FileOutputStream fileOutputStream = new FileOutputStream(file, true);  
     FileChannel fileChannel = fileOutputStream.getChannel();
     ByteBuffer byteBuffer = null;
     String messageToWrite = null;
     for(int i=1; i<limit; i++){
         //messageToWrite = get String Data From database
         byteBuffer = ByteBuffer.wrap(messageToWrite.getBytes(Charset.forName("ISO-8859-1")));
         fileChannel.write(byteBuffer);         
     }
     fileChannel.close();
}

EDIT: Tried both options. Following are the results.

@Test
public void testWritingStringToFile() {
    DiagnosticLogControlManagerImpl diagnosticLogControlManagerImpl = new DiagnosticLogControlManagerImpl();
    try {
        File file = diagnosticLogControlManagerImpl.createFile();
        long startTime = System.currentTimeMillis();
        writeToFileNIOWay(file);
        //writeToFileIOWay(file);
        long endTime = System.currentTimeMillis();
        System.out.println("Total Time is  " + (endTime - startTime));
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}

/**
 *
 * @param limit
 *            Long
 * @throws IOException
 *             IOException
 */
public void writeToFileNIOWay(File file) throws IOException {
    FileOutputStream fileOutputStream = new FileOutputStream(file, true);
    FileChannel fileChannel = fileOutputStream.getChannel();
    ByteBuffer byteBuffer = null;
    String messageToWrite = null;
    for (int i = 1; i < 1000000; i++) {
        messageToWrite = "This is a test üüüüüüööööö";
        byteBuffer = ByteBuffer.wrap(messageToWrite.getBytes(Charset
            .forName("ISO-8859-1")));
        fileChannel.write(byteBuffer);
    }
}

/**
 *
 * @param limit
 *            Long
 * @throws IOException
 *             IOException
 */
public void writeToFileIOWay(File file) throws IOException {
    FileOutputStream fileOutputStream = new FileOutputStream(file, true);
    BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(
        fileOutputStream, 128 * 100);
    String messageToWrite = null;
    for (int i = 1; i < 1000000; i++) {
        messageToWrite = "This is a test üüüüüüööööö";
        bufferedOutputStream.write(messageToWrite.getBytes(Charset
            .forName("ISO-8859-1")));
    }
    bufferedOutputStream.flush();
    fileOutputStream.close();
}

private File createFile() throws IOException {
    File file = new File(FILE_PATH + "test_sixth_one.txt");
    file.createNewFile();
    return file;
}

Using ByteBuffer and Channel: took 4402 ms

Using buffered Writer : Took 563 ms

Answer 1

UPDATED:

Since Java11 there is a specific method to write strings using java.nio.file.Files:

Files.writeString(Paths.get(file.toURI()), "My string to save");

We can also customize the writing with:

Files.writeString(Paths.get(file.toURI()), 
                  "My string to save", 
                   StandardCharsets.UTF_8,
                   StandardOpenOption.CREATE,
                   StandardOpenOption.TRUNCATE_EXISTING);

ORIGINAL ANSWER:

There is a one-line solution, using Java nio:

java.nio.file.Files.write(Paths.get(file.toURI()), 
                          "My string to save".getBytes(StandardCharsets.UTF_8),
                          StandardOpenOption.CREATE,
                          StandardOpenOption.TRUNCATE_EXISTING);

I have not benchmarked this solution with the others, but using the built-in implementation for open-write-close file should be fast and the code is quite small.

Answer 2

I don't think you will be able to get a strict answer without benchmarking your software. NIO may speed up the application significantly under the right conditions, but it may also make things slower. Here are some points:

• Do you really need strings? If you store and receive bytes from you database you can avoid string allocation and encoding costs all together.
• Do you really need rewind and flip? Seems like you are creating a new buffer for every string and just writing it to the channel. (If you go the NIO way, benchmark strategies that reuse the buffers instead of wrapping / discarding, I think they will do better).
• Keep in mind that wrap and allocateDirect may produce quite different buffers. Benchmark both to grasp the trade-offs. With direct allocation, be sure to reuse the same buffer in order to achieve the best performance.
• And the most important thing is: Be sure to compare NIO with BufferedOutputStream and/or BufferedWritter approaches (use a intermediate byte[] or char[] buffer with a reasonable size as well). I've seen many, many, many people discovering that NIO is no silver bullet.

If you fancy some bleeding edge... Back to IO Trails for some NIO2 :D.

And here is a interesting benchmark about file copying using different strategies. I know it is a different problem, but I think most of the facts and author conclusions also apply to your problem.

Cheers,

UPDATE 1:

Since @EJP tiped me that direct buffers wouldn't be efficient for this problem, I benchmark it myself and ended up with a nice NIO solution using nemory-mapped files. In my Macbook running OS X Lion this beats BufferedOutputStream by a solid margin. but keep in mind that this might be OS / Hardware / VM specific:

public void writeToFileNIOWay2(File file) throws IOException {
    final int numberOfIterations = 1000000;
    final String messageToWrite = "This is a test üüüüüüööööö";
    final byte[] messageBytes = messageToWrite.
            getBytes(Charset.forName("ISO-8859-1"));
    final long appendSize = numberOfIterations * messageBytes.length;
    final RandomAccessFile raf = new RandomAccessFile(file, "rw");
    raf.seek(raf.length());
    final FileChannel fc = raf.getChannel();
    final MappedByteBuffer mbf = fc.map(FileChannel.MapMode.READ_WRITE, fc.
            position(), appendSize);
    fc.close();
    for (int i = 1; i < numberOfIterations; i++) {
        mbf.put(messageBytes);
    }
} 

I admit that I cheated a little by calculating the total size to append (around 26 MB) beforehand. This may not be possible for several real world scenarios. Still, you can always use a "big enough appending size for the operations and later truncate the file.

UPDATE 2 (2019):

To anyone looking for a modern (as in, Java 11+) solution to the problem, I would follow @DodgyCodeException's advice and use java.nio.file.Files.writeString:

String fileName = "/xyz/test.txt";
String messageToWrite = "My long string";
Files.writeString(Paths.get(fileName), messageToWrite, StandardCharsets.ISO_8859_1);

Answer 3

A BufferedWriter around a FileWriter will almost certainly be faster than any NIO scheme you can come up with. Your code certainly isn't optimal, with a new ByteBuffer per write, and then doing pointless operations on it when it is about to go out of scope, but in any case your question is founded on a misconception. NIO doesn't 'offload the memory footprint to the OS' at all, unless you're using FileChannel.transferTo/From(), which you can't in this instance.

NB don't use a PrintWriter as suggested in comments, as this swallows exceptions. PW is really only for consoles and log files where you don't care.

Answer 4

Here is a short and easy way. It creates a file and writes the data relative to your code project:

private void writeToFile(String filename, String data) {
    Path p = Paths.get(".", filename);
    try (OutputStream os = new BufferedOutputStream(
        Files.newOutputStream(p, StandardOpenOption.CREATE, StandardOpenOption.APPEND))) {
        os.write(data.getBytes(), 0, data.length());
    } catch (IOException e) {
        e.printStackTrace();
    }
}

Answer 5

This works for me:

//Creating newBufferedWritter for writing to file
BufferedWritter napiš = Files.newBufferedWriter(Paths.get(filePath));
                    napiš.write(what);
//Don't forget for this (flush all what you write to String write):                     
                    napiš.flush();