58

Is there an efficient way to create a file with a given size in Java?

In C it can be done with ftruncate (see that answer).

Most people would just write n dummy bytes into the file, but there must be a faster way. I'm thinking of ftruncate and also of Sparse files…

Community
  • 1
  • 1
Benedikt Waldvogel
  • 12,406
  • 8
  • 49
  • 61

3 Answers3

117

Create a new RandomAccessFile and call the setLength method, specifying the desired file length. The underlying JRE implementation should use the most efficient method available in your environment.

The following program

import java.io.*;

class Test {
     public static void main(String args[]) throws Exception {
           RandomAccessFile f = new RandomAccessFile("t", "rw");
           f.setLength(1024 * 1024 * 1024);
     }
}

on a Linux machine will allocate the space using the ftruncate(2)

6070  open("t", O_RDWR|O_CREAT, 0666)   = 4
6070  fstat(4, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
6070  lseek(4, 0, SEEK_CUR)             = 0
6070  ftruncate(4, 1073741824)          = 0

while on a Solaris machine it will use the the F_FREESP64 function of the fcntl(2) system call.

/2:     open64("t", O_RDWR|O_CREAT, 0666)               = 14
/2:     fstat64(14, 0xFE4FF810)                         = 0
/2:     llseek(14, 0, SEEK_CUR)                         = 0
/2:     fcntl(14, F_FREESP64, 0xFE4FF998)               = 0

In both cases this will result in the creation of a sparse file.

demongolem
  • 9,474
  • 36
  • 90
  • 105
Diomidis Spinellis
  • 18,734
  • 5
  • 61
  • 83
  • How did you record these library call traces? ^^ – akuhn Oct 29 '08 at 00:38
  • 2
    They are system call traces. I used strace(1) under Linux and truss(1) under Solaris. – Diomidis Spinellis Oct 29 '08 at 06:37
  • In javadoc, it is written that "In this case, the contents of the extended portion of the file are not defined". Does this still guaranties to be zeroed on windows and linux, or is there any efficient way to be sure all bytes are zeros? – Sarmun Mar 27 '09 at 23:15
  • 1
    I would expect that any OS worth its salt (this includes Windows and Linux) to zero the bytes, to avoid the leakage of older data belonging to another user. However, I can think of scenarios where this would not be the case: old data belonging to the same process, or a small (J2ME) platform. – Diomidis Spinellis Mar 29 '09 at 13:00
  • 1
    Brilliant. I used it to test an Android app under "SD card filled" conditions! – Mauro Vanetti Mar 18 '11 at 17:08
  • Interestingly `RandomAccessFile.setLength()` API also fills the file with zero's which can be very useful at times. – Sandeep Jul 14 '11 at 18:21
  • 3
    @Sandeep that is not correct. The Javadoc specifically says that 'the contents of the extended portion of the file are not defined'. The zeros may be the behaviour on a specific platform. – user207421 Oct 18 '11 at 00:52
  • Does the above answer really work? The only thing created on my test system (a CentOS 5.6 VM) seems to be an EMPTY file that reports the "allocated" size via the length() method, which can be arbitrarily large, irrespective of available space. If you open the file and write a few characters, the file size changes to a few bytes. The getFreeSpace() method of the File class reports the same amount of available space before and after "creating" the file and you can use up the free disk space of the partition as if the created file was not there, irrespective of how large the preset length is. – PNS Oct 21 '11 at 01:12
  • A file of a given size, does not have to occupy that many bytes. Many systems support sparse files (this is what you are seeing), where parts of the file that are empty are not stored on disk. Other systems may support compressed files, where the amount of storage used depends on how compressible are the file's contents. – Diomidis Spinellis Jul 12 '14 at 14:49
  • Works as it is expected, but are there any ways to configure this approach to generate unique files (which MD5 values are different)? @DiomidisSpinellis – talha06 Feb 14 '17 at 11:38
  • To make the files uniquely different create them with a different length, or tack a UUID at the beginning or at the end of the file. – Diomidis Spinellis Feb 16 '17 at 08:22
  • I think this is a bad solution, the size on disk won't match the file length you want. – Kreender Jun 04 '20 at 20:40
  • The original question specifically asked about creating a sparse file. Sparse files by definition require less storage than their length. – Diomidis Spinellis Jun 05 '20 at 21:05
5

Since Java 8, this method works on Linux and Windows :

final ByteBuffer buf = ByteBuffer.allocate(4).putInt(2);
buf.rewind();

final OpenOption[] options = { StandardOpenOption.WRITE, StandardOpenOption.CREATE_NEW , StandardOpenOption.SPARSE };
final Path hugeFile = Paths.get("hugefile.txt");

try (final SeekableByteChannel channel = Files.newByteChannel(hugeFile, options);) {
    channel.position(HUGE_FILE_SIZE);
    channel.write(buf);
}
mandev
  • 51
  • 1
  • 2
4

You can open the file for writing, seek to offset (n-1), and write a single byte. The OS will automatically extend the file to the desired number of bytes.

Greg Hewgill
  • 951,095
  • 183
  • 1,149
  • 1,285