3

I am polling file system for new file, which is upload by someone from web interface. Now I have to process every new file, but before that I want to insure that the file I am processing is complete (I mean to say it is completely transferred through web interface).

How do I verify if file is complete downloaded or not before processing?

BalusC
  • 1,082,665
  • 372
  • 3,610
  • 3,555
Dhruv
  • 10,291
  • 18
  • 77
  • 126

2 Answers2

8

Renaming a filename is an atomic action in most (if not all) filesystems. You can make use of this by uploading the file to a recognizable temporary name and renaming it as soon as the upload is complete.

This way you will "see" only those files that have been uploaded completely and are safe for processing.

BalusC
  • 1,082,665
  • 372
  • 3,610
  • 3,555
rsp
  • 23,135
  • 6
  • 55
  • 69
  • 2
    This isnt really a hack. Unless the uploading process locks the file or the file has a recognisable end-of-file marker this is the ONLY way to do this and is standard practise. Even browsers such as firefox do the same thing (create a hidden *.part file and then rename to the expected target file). – Philip Couling Dec 23 '11 at 14:39
  • Whether there is or there isn't another way to do it, whether it is standard practice or not, etc. all these have nothing to do with whether it is a hack or not. It is a hack, because the non-hacky way of doing it would be to ask the operating system "is this file opened by someone else?". Nonetheless, it is definitely a case of a good hack. – Mike Nakis Dec 23 '11 at 14:57
  • Sadly a large number of people are using an OS with non atomic renames (every windows before Vista, ie XP) and I don't know whether the Java APIs take advantage of the fact that it is possible to do atomic renames in modern Windowses. But then the only real alternative would be opening the file exclusively and does java even allow for that? – Voo Dec 23 '11 at 15:00
  • Yes, I think it does. (And that's what my answer proposes.) Here is a question in which this has been answered (though not accepted): http://stackoverflow.com/questions/1390592/java-check-if-file-is-already-open – Mike Nakis Dec 23 '11 at 21:08
2

rsp's answer is very good. If, by any chance, it does not work for you, and if your polling code is running within a process different from the process of the web server which is saving the file, you might want to try the following:

Usually, when a file is being saved, the sharing options are "allow anyone to read" and "allow no-one to write". (exclusive write.) Therefore, you can attempt to open the file also with exclusive write access: if this fails, then you know that the web server is still holding the file open, and writing to it. If it succeeds, then you know that the web server is done. Of course be sure to try it, because I cannot guarantee that this is precisely how the web server chooses to lock the file.

Mike Nakis
  • 56,297
  • 11
  • 110
  • 142
  • I am using the linux system. I have done simple experiment, I have tried making File object of one file which is not completed downloaded, through java I was able to process the file. What I want is some method to check whether file is completely downloaded or not before accessing and processing the file. Tell me if i am wrong , I am just trying to simulate torrent file case with my actual case which I am facing. In my project there will be big pdf files upload by some third person, so before processing them I just wanted to insure that, that files are completely uploaded to my server. – Dhruv Dec 25 '11 at 11:42
  • Well, if you tried opening the file with exclusive write and you succeeded, this means that your torrent client does not open the same file with exclusive write. This does not mean that your web server works the same way; on your web server it may succeed. – Mike Nakis Dec 25 '11 at 11:44