4

Anybody know how to execute methods (python) when files are modified like Dropbox and his Continuous Data Protection mechanism that can track exactly when a file is modified and sync it.

Of course it would not be of the entire hard-disk, but a track on a specified directory.

OBS: For Windows and Linux OS. Mac is a plus ;)

Pabluez
  • 2,653
  • 3
  • 19
  • 29

3 Answers3

5

On Linux, pyinotify will probably do what you want. But note the caveats mentioned in the inotify(7) manpage, in particular:

Note that the event queue can overflow. In this case, events are lost. Robust applications should handle the possibility of lost events gracefully.

If monitoring an entire directory subtree, and a new subdirectory is created in that tree, be aware that by the time you create a watch for the new subdirectory, new files may already have been created in the subdirectory. Therefore, you may want to scan the contents of the subdirectory immediately after adding the watch.

Community
  • 1
  • 1
Thomas
  • 174,939
  • 50
  • 355
  • 478
  • Amazing module! I think is that thing that i should use for linux. It does apparently what i need. I'm gonna give it a try and report ASAP. – Pabluez Dec 10 '11 at 20:22
  • referring to the other comment, that's what i was looking for. Pyinotify handles the kernel "bult-in" inotify which is the responsible for tracking file system on linux since kernel 2.6.13. Well, thanks a lot. – Pabluez Dec 11 '11 at 00:15
  • 1
    Be aware though that inotify has no guarantee that an event will reach your code. There are situations during reboot, restart of services and under high load where events can get lost. So if you're going to do reliable sync/replication you'll need to keep some sort of database of what has been synced, and scan all the files for changes. Probably using checksums. Inotify can be a nice addition to get instant sync, but you'll need the scan routines too. – Anders S Jun 22 '17 at 16:52
4

I'm not sure if Python has any cross-platform solution for this, but if you are only interested in Windows-based solutions, you should look into directory change notifications. To call the Win32 API functions, you can look into pywin32.

On Linux, there seems to be a bunch of solutions, including fschange, dnotify and inotify. I'm not sure which one is the recommended solution, but inotify seems to be the most complete solution.

Not all platforms have such a feature. If it's not available for a given platform, you'll have to emulate such notifications by checking directory contents periodically.

André Caron
  • 44,541
  • 12
  • 67
  • 125
  • Great answer! And do you know how could it be on linux OS? – Pabluez Dec 10 '11 at 19:21
  • I'm gonna see those solutions and report ASAP. – Pabluez Dec 10 '11 at 20:23
  • yeh, fschange is the old dnotify which also is the old inotify. Inotify is part of the kernel since 2.6.13. This suppose to be used in C. For Python, there is a module called Pyinotify that handle generically the "api" inotify and provide what i need. (the user @Thomas provided this information). So, it's is exactly what i was looking for, but you provided the informations for windows and that's very useful as well for future reference. thank's a lot. – Pabluez Dec 11 '11 at 00:13
0

What you need is rsync. There are several implementation of rsync in python. Check these out -

http://pypi.python.org/pypi/rsync.py/2.0

http://code.activestate.com/recipes/577518-rsync-algorithm/

Looking for cross-platform rsync-like functionality in python, such as rsync.py

Controlling rsync with Python?

Community
  • 1
  • 1
Srikar Appalaraju
  • 71,928
  • 54
  • 216
  • 264
  • Isn't Rsync only for backup ? I do not want to backup or sync anything, i wanna only track the files and execute my methods whenever those files are modified. – Pabluez Dec 10 '11 at 19:30
  • rsync makes sure 2 or more locations have syncronised data. It does this by tracking changes & syncing only those delta changes. So all those locations are as close to similarity as possible. Isn't that what you need ? – Srikar Appalaraju Dec 10 '11 at 19:37
  • Yeah, this common behavior of the rsync I know well. What I really want is to track modifications to generate alerts and do some other taks (logging, rootkit verification, etc). But not to have any copy of the file somewhere else. It would be great if rsync could accomplish this task, as it runs on many platforms. – Pabluez Dec 10 '11 at 19:41