11

WatchService sounded like an exciting idea ... unfortunately it seems to be as low-level as warned in the tutorial/api plus doesn't really fit into the Swing event model (or I'm missing something obvious, a not-zero probability

Taking the code from WatchDir example in the tutorial (simplyfied to handle a single directory only), I basically ended up

  • extend SwingWorker
  • do the registration stuff in the constructor
  • put the endless loop waiting for a key in doInBackground
  • publish each WatchEvent when retrieved via key.pollEvents()
  • process the chunks by firing propertyChangeEvents with the deleted/created files as newValue

    @SuppressWarnings("unchecked")
    public class FileWorker extends SwingWorker<Void, WatchEvent<Path>> {
    
        public static final String DELETED = "deletedFile";
        public static final String CREATED = "createdFile";
    
        private Path directory;
        private WatchService watcher;
    
        public FileWorker(File file) throws IOException {
            directory = file.toPath();
            watcher = FileSystems.getDefault().newWatchService();
            directory.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
        }
    
        @Override
        protected Void doInBackground() throws Exception {
            for (;;) {
                // wait for key to be signalled
                WatchKey key;
                try {
                    key = watcher.take();
                } catch (InterruptedException x) {
                    return null;
                }
    
                for (WatchEvent<?> event : key.pollEvents()) {
                    WatchEvent.Kind<?> kind = event.kind();
                    // TBD - provide example of how OVERFLOW event is handled
                    if (kind == OVERFLOW) {
                        continue;
                    }
                    publish((WatchEvent<Path>) event);
                }
    
                // reset key return if directory no longer accessible
                boolean valid = key.reset();
                if (!valid) {
                    break;
                }
            }
            return null;
        }
    
        @Override
        protected void process(List<WatchEvent<Path>> chunks) {
            super.process(chunks);
            for (WatchEvent<Path> event : chunks) {
                WatchEvent.Kind<?> kind = event.kind();
                Path name = event.context();
                Path child = directory.resolve(name);
                File file = child.toFile();
                if (StandardWatchEventKinds.ENTRY_DELETE == kind) {
                    firePropertyChange(DELETED, null, file);
                } else if (StandardWatchEventKinds.ENTRY_CREATE == kind) {
                    firePropertyChange(CREATED, null, file);
                }
            }
        }
    
    }
    

The basic idea is to make using code blissfully un-aware of the slimy details: it listens to the property changes and f.i. updates arbitrary models as appropriate:

    String testDir = "D:\\scans\\library";
    File directory = new File(testDir);
    final DefaultListModel<File> model = new DefaultListModel<File>();
    for (File file : directory.listFiles()) {
        model.addElement(file);
    }
    final FileWorker worker = new FileWorker(directory);
    PropertyChangeListener l = new PropertyChangeListener() {

        @Override
        public void propertyChange(PropertyChangeEvent evt) {
            if (FileWorker.DELETED == evt.getPropertyName()) {
                model.removeElement(evt.getNewValue());
            } else if (FileWorker.CREATED == evt.getPropertyName()) {
                model.addElement((File) evt.getNewValue());
            }
        }
    };
    worker.addPropertyChangeListener(l);
    JXList list = new JXList(model);

Seems to work, but I feel uncomfortable

  • Outing myself as the thread agnostic I am: all example snippets I have seen so far do block the waiting thread by using watcher.take(). Why do they do it? Would expect at least some use watcher.poll() and sleep a bit.
  • the SwingWorker publish method doesn't quite seem to fit: for now it's okay, as I'm watching one directory only (didn't want to galopp too far into the wrong direction :) When trying to watch several directories (as in the original WatchDir example) there are several keys and the WatchEvent relative to one of those. To resolve the path, I would need both the event and the directory [A] the key is watching - but can pass on only one. Most probably got the distribution of logic wrong, though

[A] Edited (triggered by @trashgods's comment) - it's actually not the key I have to pass around along with the event, it's the directory it's reporting the changes on. Changed the question accordingly

FYI, this question is cross-posted to the OTN swing forum

Addendum

Reading the api doc of WatchKey:

Where there are several threads retrieving signalled keys from a watch service then care should be taken to ensure that the reset method is only invoked after the events for the object have been processed.

seems to imply that the events should

  1. be processed on the same thread that retrieved the WatchKey
  2. shouldn't be touched after the key is reset

Not entirely sure, but combined with the (future) requirement to recursively watching directories (more than one) decided to follow @Eels advice, kind of - will soon post the code I settled on

EDIT just accepted my own answer - will humbly revert that if anybody has reasonable objections

Roman C
  • 49,761
  • 33
  • 66
  • 176
kleopatra
  • 51,061
  • 28
  • 99
  • 211
  • 1
    With regard to your second point, couldn't you create a class that holds both WatchEvent and key and have the SwingWorker's second generic parameter be this type? Sorry, I know you've already thought of this, so I guess my question is: any down-side to doing this? – Hovercraft Full Of Eels Oct 16 '11 at 14:58
  • @Hovercraft Full Of Eels actually nothing but the bad feeling here, it would be a mere struct, and when I think I need something like such a struct my alarm bells start ringing ... so I'm likely doing something wrong :-) – kleopatra Oct 16 '11 at 15:14
  • +1 for question, let's Java Essential Classes managed java.util.Xxx, and (if isn't there another way) for input/output to these processes use SwingWorker or simple javax.swing.Actions do that better and without shadowed bridge to the Swing_to_Future – mKorbel Oct 16 '11 at 19:42
  • @Hovercraft Full Of Eels consider to make your advice an answer, so I can vote for it - it was the trigger for my solution :-) – kleopatra Oct 21 '11 at 11:53
  • @kleo: Done, thanks. Please be sure to flag trash and me once you post the final code! – Hovercraft Full Of Eels Oct 21 '11 at 11:56

3 Answers3

4

Because your background thread is devoted entirely to watching, take() is the right choice. It effectively hides the platform dependent implementation, which may either forward or poll. One of the poll() methods would be appropriate if, for example, your background thread also needed to examine other queues in series with the WatchService.

Addendum: Because the WatchKey has state, it should probably not be forwarded to process(). The context() of a WatchEvent is a "relative path between the directory registered with the watch service and the entry that is created, deleted, or modified." One of the resolve() methods should work if the directories share a common root.

trashgod
  • 203,806
  • 29
  • 246
  • 1,045
  • Disclaimer: I haven't tried this, but the pattern is familiar. – trashgod Oct 16 '11 at 17:00
  • You're welcome. I've elaborated above, but file systems with more than one root are _terra incognita_ to me. I'd welcome a more dispositive answer if you experiment more or get something on OTN. – trashgod Oct 16 '11 at 19:47
  • good point regarding the key. Turns out, I dont need it but the directory it is reporting the change on. Edited the question, thanks – kleopatra Oct 17 '11 at 08:11
  • thanks, helped me come up with the (final? at least currently good enough :) answer – kleopatra Oct 21 '11 at 12:19
4

Actually, @Eels's comment didn't stop knocking in the back of my head - and finally registered: it's the way to go, but there is no need for any "artificial" struct, because we already have the perfect candidate - it's the PropertyChangeEvent itself :-)

Taking the overall process description from my question, the first three bullets remain the same

  • same: extend SwingWorker
  • same: do the registration stuff in the constructor
  • same: put the endless loop waiting for a key in doInBackground
  • changed: create the appropriate PropertyChangeEvent from each WatchEvent when retrieved via key.pollEvents and publish the PropertyChangeEvent
  • changed: fire the previously created event in process(chunks)

Revised FileWorker:

@SuppressWarnings("unchecked")
public class FileWorker extends SwingWorker<Void, PropertyChangeEvent> {

    public static final String FILE_DELETED = StandardWatchEventKinds.ENTRY_DELETE.name();
    public static final String FILE_CREATED = StandardWatchEventKinds.ENTRY_CREATE.name();
    public static final String FILE_MODIFIED = StandardWatchEventKinds.ENTRY_MODIFY.name();

    // final version will keep a map of keys/directories (just as in the tutorial example) 
    private Path directory;
    private WatchService watcher;

    public FileWorker(File file) throws IOException {
        directory = file.toPath();
        watcher = FileSystems.getDefault().newWatchService();
        directory.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
    }

    @Override
    protected Void doInBackground() throws Exception {
        for (;;) {
            // wait for key to be signalled
            WatchKey key;
            try {
                key = watcher.take();
            } catch (InterruptedException x) {
                return null;
            }

            for (WatchEvent<?> event : key.pollEvents()) {
                WatchEvent.Kind<?> kind = event.kind();
                // TBD - provide example of how OVERFLOW event is handled
                if (kind == OVERFLOW) {
                    continue;
                }
                publish(createChangeEvent((WatchEvent<Path>) event, key));
            }

            // reset key return if directory no longer accessible
            boolean valid = key.reset();
            if (!valid) {
                break;
            }
        }
        return null;
    }

    /**
     * Creates and returns the change notification. This method is called from the 
     * worker thread while looping through the events as received from the Watchkey.
     * 
     * @param event
     * @param key
     */
    protected PropertyChangeEvent createChangeEvent(WatchEvent<Path> event, WatchKey key) {
        Path name = event.context();
        // real world will lookup the directory from the key/directory map
        Path child = directory.resolve(name);
        PropertyChangeEvent e = new PropertyChangeEvent(this, event.kind().name(), null, child.toFile());
        return e;
    }

    @Override
    protected void process(List<PropertyChangeEvent> chunks) {
        super.process(chunks);
        for (PropertyChangeEvent event : chunks) {
            getPropertyChangeSupport().firePropertyChange(event);
        }
    }
}
trashgod
  • 203,806
  • 29
  • 246
  • 1,045
kleopatra
  • 51,061
  • 28
  • 99
  • 211
  • you cann't use char ' `-` ' before code block, I put here dot as `separator` – mKorbel Oct 21 '11 at 15:03
  • ahh ... hidden rules, how I _love_ them ;-) thanks @trashgod, @mKorbel! – kleopatra Oct 21 '11 at 21:40
  • My requirement is to watch a folder and as soon as a file is added/written/moved into the folder, take an action immediately on it (for e.g. email the file). Problem I am running into is that when the file is large, it may take a while to get completed written or copied, whereas the FILE_CREATE event is announced as soon as the first bytes of the file get written into the folder. So I cannot perform the action immediately. What is a reliable way to determine if the file has been fully written before performing any action on it? – Web User Oct 28 '13 at 14:05
  • @WebUser no idea, sorry - may be you get a modified at the end? – kleopatra Oct 29 '13 at 09:51
  • Ahh nuts. I came here because the OVERFLOW event is causing us grief and nobody is really sure how to implement it correctly. I do know that ignoring it and continuing the loop is no good, because you lose updates. But resetting the state and beginning from the start doesn't seem to be right either, because now we somehow have multiple copies of the same files. It seems like it might be really hard to implement correctly. – Hakanai Feb 06 '14 at 00:09
  • @Trejkaz maybe (just guessing here) there is _nothing_ that could be done? For the standard implementation, the context isn't specified for overflow. Curious though: when do you experience it? what _is_ the context in such case? Maybe you can store it in some internal list and go back to it when time permits (whatever that could mean ;-) – kleopatra Feb 06 '14 at 10:58
  • 1
    The only evidence we have of it is in customer logs. I haven't seen it myself yet. But it seems like, yes... when you get an overflow, the correct behaviour is to shut down everything and start crawling the file tree all over again. The sad part of all this for me is that the shiny new API is so full of holes that by the time you have plugged all the holes, you might as well have just crawled the directory. Plus depending on the platform, the implementation may be naively crawling the directory anyway! – Hakanai Feb 07 '14 at 04:39
  • 1
    By the way, I was ready to accept this overflow thing as "an OS-specific issue where sometimes the OS won't have enough space to store the events", until reading the source, only to discover that the code introducing the limit is in the JDK itself. Hard-coded to only 512 events - and not customisable in any way whatsoever. – Hakanai Feb 07 '14 at 04:40
  • @Trejkaz the joys of hard-codeing anything ... :-( Thanks for the info – kleopatra Feb 07 '14 at 09:23
3

With regard to your second point, couldn't you create a class that holds both WatchEvent and key and have the SwingWorker's second generic parameter be this type? Sorry, I know you've already thought of this, so I guess my question is: any down-side to doing this?

Hovercraft Full Of Eels
  • 283,665
  • 25
  • 256
  • 373