0

I've a scenario where we have spring intergration file poller, waiting for files to be added to a directory, once written we process the file. We have a some large files and slow network so i some cases we are worried that the file transfer may not have finished when the poller wakes and attempts to process the file.

I've found this topic on 'file-inbound-channel-unable-to-read-file' which suggest using a custom filter to check if the file is readable before processing.

This second topic 'how-to-know-whether-a-file-copying-is-in-progress-complete' suggest that the file must be writable before it can be considered to be ready for processing.

I might have expected that this checking that the file is read/writable would already be done by the core spring intergration code?.

In the mean time i'm planning on creating a filter as per the first topic, but using the 'rw' check that the second suggests.

Community
  • 1
  • 1
emeraldjava
  • 10,894
  • 26
  • 97
  • 170

1 Answers1

2

This is a classic problem and just checking if the file is writable is not reliable - what happens if the network crashed during the file transfer? You will still have an incomplete file.

The most reliable way to handle this is to have the sender send the file with a temporary suffix and rename it after the transfer is complete. Another common technique is to send a second file foo.done indicating that foo.xxx is complete, and use a custom filter for that.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179