FileUtils.moveFile not always throwing IOException when disk is full

Question

FileUtils.moveFile is not always throwing IOException when it fails to move the fail in the event of disk full situation. The new file gets created but the size is 0. Below is the code

void  function1 (String filePath1, String dir) 
{
    File file1= new File( filePath1);
    File file2= new File( dir, file1.getName() );
    
    FileUtils.moveFile( file1, file2);
}

Multiple threads are using above function. Every thread has a unique filePath1 value. The disk storage is controlled by a NFS server. So some of the threads are throwing following exception (file names have been masked) when move file fails and empty file is created.

Failed to copy full contents from '######' to '######' Expected length: 239 ?Actual: 0

at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:1164)

at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1088)

at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1040)

at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2993)

at function1

But some of them do not throw any exception, and destination file is empty.

To verify this I added check for file size.

void  function1 (String filePath1, String dir)
{
    File file1= new File( filePath1);
    File file2= new File( dir, file1.getName() );
    long sizeOfFile1 = FileUtils.sizeOf( file1);
    FileUtils.moveFile( file1, file2);
    long sizeOfFile2 = FileUtils.sizeOf( file2);

    if( sizeOFile1 != sizeOfFile2 )
          LOG.error( "###### expected size " + sizeOFile1 + " but actual size " +  sizeOfFile2 + " for thread " + ######## );
        throw new IOException();

    }
}

After this every thread that was not throwing exception earlier started printing the error in above function

expected size 239 but actual size 0 for thread

So I wanted to understand if there is something else that could have been done in case of NFS storage or is there a bug in JVM due to which it doesn’t inform the application when it has failed to move the file?

Check this: http://stackoverflow.com/questions/1051295/how-to-find-how-much-disk-space-is-left-using-java `File` class has some methods to check diskspace — Roel Strolenberg, Jan 13 '17 at 07:42
Thanx for the link. But the objective of the program is not to detect when the disk is full. The problem here is FileUtils.moveFile function is not always throwing the IOException when it fails to move the file. In one thread it throws the exception and in another it does not. In both threads the destination file is empty. — Jasprit Singh, Jan 13 '17 at 09:05
@rkosegi: In the description of "FileUtils.moveFile" it is written "IOException - if an IO error occurs moving the file" and which it is throwing in one of the threads. Same is written in description of java.nio.file.Files.move function " IOException - if an I/O error occurs". So i want to understand why it is not always working. — Jasprit Singh, Jan 13 '17 at 09:13
Which version of FileUtils is it? Can you please specify the maven dependency? For example, `org.apache.commons commons-io 1.3.2` provides `FileUtils.copyFile( file1, file2)`, which compares the file sizes and may throw an IOException accordingly. — SME_Dev, Jan 13 '17 at 10:00

Stephen C · Answer 1 · 2017-01-13T14:09:37.093

The normal behavior for a write(...) syscall to a full filesystem is to fail and set errno to ENOSPC. The Java would then turn that into an IOException.

However, it appears that your NFS mount is not reporting file-system-full like a normal file system would. Here is some evidence that this kind of thing can happen:

NFS behaviour when filesystem is 100% full

If the write syscall is not reporting an error, there is nothing that the Java runtime can do to detect the problem. If this is what is happening, then it is not a Java bug. It is probably a bug in the implementation of the NFS server or client-side implementation that you are using.

I would expect the FileUtils.moveFile and copyFile methods to detect the problem. However you didn't say which version of Apache Commons you are using, so I can't be sure of that. Also, since you are using multiple threads to move files, it is possible that two threads are trying to move the same file, and that that is causing problems.

Interesting writeup of NFS issues. Too bad it's based on NFS mounting of files shared from `localhost` - something that's not really supported if you were to report it as a bug. — Andrew Henle, Jan 13 '17 at 16:57

FileUtils.moveFile not always throwing IOException when disk is full

1 Answers1