java io append to file implementation

Question

I'm using java-1.8.0-openjdk-amd64 over Ubuntu 16.04

I'm trying to understand what happens to file while I append

Lets assume I'm appending to a very large file, does Java loads the whole file into memory ? how can I see the native calls ?

From java.io.FileOutputStream.java

/**
 * Opens a file, with the specified name, for overwriting or appending.
 * @param name name of file to be opened
 * @param append whether the file is to be opened in append mode
 */
private native void open0(String name, boolean append)
    throws FileNotFoundException;

// wrap native call to allow instrumentation
/**
 * Opens a file, with the specified name, for overwriting or appending.
 * @param name name of file to be opened
 * @param append whether the file is to be opened in append mode
 */
private void open(String name, boolean append)
    throws FileNotFoundException {
    open0(name, append);
}

Update:

Looking at jdk8 source code

src/solaris/native/sun/nio/ch/InheritedChannel.c:Java_sun_nio_ch_InheritedChannel_open0(JNIEnv *env, jclass cla, jstring path, jint oflag)

JNIEXPORT jint JNICALL
Java_sun_nio_ch_InheritedChannel_open0(JNIEnv *env, jclass cla, jstring path, jint oflag)
{
    const char* str;
    int oflag_actual;

    /* convert to OS specific value */
    switch (oflag) {
        case sun_nio_ch_InheritedChannel_O_RDWR :
            oflag_actual = O_RDWR;
            break;
        case sun_nio_ch_InheritedChannel_O_RDONLY :
            oflag_actual = O_RDONLY;
            break;
        case sun_nio_ch_InheritedChannel_O_WRONLY :
            oflag_actual = O_WRONLY;
            break;
        default :
            JNU_ThrowInternalError(env, "Unrecognized file mode");
            return -1;
    }

    str = JNU_GetStringPlatformChars(env, path, NULL);
    if (str == NULL) {
        return (jint)-1;
    } else {
        int fd = open(str, oflag_actual);
        if (fd < 0) {
            JNU_ThrowIOExceptionWithLastError(env, str);
        }
        JNU_ReleaseStringPlatformChars(env, path, str);
        return (jint)fd;
    }
}

https://stackoverflow.com/questions/12594046/java-native-method-source-code — Tom, Sep 04 '17 at 09:32

Kayaman · Accepted Answer · 2017-09-04T09:45:04.807

2

No, it doesn't load the file into memory. That would make it impossible to append to large files.

The actual logic depends on the file system being used, but basically only the last block of the file needs to be loaded, rewritten with appended data and additional blocks written for any further appended data.

You don't need to worry about it. If you do want to worry about it, learn how file systems work.

edited Sep 04 '17 at 09:45

answered Sep 04 '17 at 09:34

Kayaman

72,141
5
83
121

I would like to know which sys calls java invokes so I could learn about them and also understand if the JVM loads the "last block" is that means the last row ? if I have a 1GB file with only 5 rows ? it loads 250mb block to the RAM ? – whoopdedoo Sep 04 '17 at 09:49
1

Read about how file systems work. Java has nothing to do with this, and blocks aren't rows. – Kayaman Sep 04 '17 at 09:50
+1 but I can not accept as answer, please provide reference related to append vs write to file ( if I may ask for that ) – whoopdedoo Sep 04 '17 at 11:21
Here you go: https://en.wikipedia.org/wiki/File_system Make sure to compare the intricacies between different file systems, like `NTFS`, `ext4` and why not `ZFS` as well. – Kayaman Sep 04 '17 at 11:47
would be glad to get explanation about the "block" it load to memory on append, well.. answer accepted. thanks – whoopdedoo Sep 04 '17 at 12:32
It's way too broad to explain here. If you're interested, spend a few days reading, if not, don't expect anyone to give you a satisfying answer in few minutes. Computers are a lot more complicated than a lot of younger people understand. I guess you're pretty shielded from the nuts and bolts these days. – Kayaman Sep 04 '17 at 12:35
English is not my native language so neither I nor google's first page results understand what "pretty shielded from the nuts and bolts these days" means, but I'm working on very complicated system with I/O bottleneck.. well I will leave this dialog before it became too philosophical, if you have any good reference beside wikipedia, please share – whoopdedoo Sep 04 '17 at 12:45
1

I meant you don't understand how computers and operating systems work on the lower level. Nothing weird about that, but it's naive to assume that you can get years worth of knowledge just by asking someone to explain things. – Kayaman Sep 04 '17 at 12:49

java io append to file implementation

1 Answers1