13

I'm trying to write the fastest, most optimal file saving method possible. Is there any way to get the system block size in java? Something like System.getProperty("block.size") or something.

kentcdodds
  • 27,113
  • 32
  • 108
  • 187
  • 1
    There is no System property for this, and no way to get the information thru pure java. If this is such a concern you should use a BufferedWriter or a BufferedOutputStream – ControlAltDel May 07 '12 at 14:52
  • 2
    +1; good question. But IMO getting the storage block size from the System environment does not make sense as you may have several mountpoints on a given system backed by different types of storage. – home May 07 '12 at 14:52
  • 9
    This is why they build operating systems - to spare you from these details – ControlAltDel May 07 '12 at 14:52
  • fyi: http://stackoverflow.com/questions/3892186/how-can-the-block-size-of-a-file-store-be-portably-obtained-in-java-7 – home May 07 '12 at 14:55
  • 1
    @ControlAltDel I guess the abstraction is leaky then, because even if you use BufferedWriter, you still have to choose a buffer size, which then comes back to the same question of how to get the block size for that file. – Hakanai Apr 01 '14 at 02:02

4 Answers4

6

I don't think there is a java function to do that. However there is a system dependent way to retrieve that information. Have a look at this answer. You'll need to use Runtime.exec() and parse the data.

Community
  • 1
  • 1
user845279
  • 2,794
  • 1
  • 20
  • 38
6

The 'fastest most optimal way possible' is surely to write using the biggest buffer you can afford; to make sure its size is a power of two (a megabyte comes to mind); and to make sure that the writes themselves are buffer-aligned:

new BufferedOutputStream(new FileOutputStream(file), 1024*1024);

As long as you are over the system block size, which you will be at this size, and remain aligned with it, which is guaranteed by the BufferedOutputStream, this is about as optimal as it gets.

You should also look into FileChannel.transferTo(), noting that you must call it in a loop, and that the actual implementations so far don't appear to use any low-level operating system primitives (contrary to the advertising), just the same kind of loop you could write yourself.

user207421
  • 305,947
  • 44
  • 307
  • 483
  • I did some banchmarking on a mobile phone (Nexus 5X) for my Android app for both: small files (3,5Mb) and large files (175 Mb). And found out that the golden size would be byte[] of 524288 lengths. Well, you may win 10-20ms if you switch between small buffer 4Kb and big buffer 524Kb depending on the file size but it's not worth it. Just stick with 524 Kb. – Kirill Karmazin Oct 25 '18 at 12:31
4

As of Java 9, you can use the FileSystem and FileStore class to get the system disk block size based on the file system implemented by your native OS. The code is as follows:

    FileSystem fs = FileSystems.getDefault();
    Iterable<Path> roots = fs.getRootDirectories();
    for (FileStore store: fs.getFileStores()) {
        Iterator<Path> i = roots.iterator();
        while(i.hasNext()) {
            if(store.name().startsWith(i.next().toString())) {
                System.out.println("Name: " + store.name() + ", Read only: " + store.isReadOnly() + ", Type: " + store.type() + ", Block: " + store.getBlockSize() + " Bytes");
            }
        }
    }
Anirban Das
  • 71
  • 2
  • 6
  • Your code creates bugs on Linux because there is always at least one root, *the* root (`/`), that starts all paths. It's better to retrieve the appropriate `FileStore` using `Files.getFileStore(path)` which does the correct computation for you. – Olivier Grégoire Feb 06 '22 at 21:57
3

There's no System property but you can use whatever Java uses for its own needs and rest assured that that's the best for your OS/Java-version combo.

This solution exploits the fact the buf used in Java's BufferedXXXStream classes to hold the data is a protected variable and not a private one:

class IdealBlockSize {
    // You could alternatively use BufferedInputStream and System.in .
    private static class MyBufferedOS extends BufferedOutputStream {
        public MyBufferedOS() { super(System.out); }
        public MyBufferedOS(OutputStream out) { super(out); }
        public int bufferSize() { return buf.length; }
    }

    public static int VALUE = new IdealBlockSize.MyBufferedOS().bufferSize();
}

Then, you can invoke them in your application startup code, like so:

System.out.printf("Ideal block size: %d\n", IdealBlockSize.VALUE);

// Output
// Ideal block size: 8192

This solution allows you to have buffers that are just big enough to be optimal.

Harry
  • 3,684
  • 6
  • 39
  • 48
  • 2
    You get 8192 because that default value is hard coded in the BufferedOutputStream constructor. No detection is done. I believe the OP is looking for a solution that optimises his buffer size based upon that of the underlying block device. If his block device is (as is common) 4k aligned then this will most certainly not be optimal. – Dave Dec 27 '16 at 05:00
  • The choice of 8192 was likely a compromise across the orthogonal requirements or read vs write and sequential vs random access and varying operating system choices. OP wants to write fast. Depending on why, direct IO might even be best and if using sync then the logical block size of the FS might be optimal. Most people's experience of setting buffer size comes from copying, where most of the clock time is actually used in reads. In that case much larger buffers help due to the OS read-ahead buffer. This is not the case for write-only. – Dave Dec 27 '16 at 06:19