1

I have a mail parser perl script which is called every time a mail arrives for a user (using .qmail). It extracts a calendar attachment out of the mail and places the "path" of the file in a FIFO queue implemented using the Directory::Queue module.

Another perl script which reads the path of the calendar attachment and performs certain file operations on the local system as well as on the remote CalDAV server, is being run as a daemon, as explained here. So basically this script looks like:

my $declarations

sub foo {
.
.
}

sub bar {
. 
. 
}

while ($keep_running) {
    for(keep-checking-the-queue-for-new-entries) {

        sub caldav_logic1 {
        .
        .
     }
        sub caldav_logic2 {
        .
        . 
    }
  }
}

I am using Proc::Daemon for running the script as a daemon. Now the problem is, this process has almost 100% CPU usage. What are the suggested ways to implement the daemon in a more standard, safer way ? I am using pretty much the same code as mentioned in the link mentioned for usage of Proc::Daemon.

Community
  • 1
  • 1
Mohit Chawla
  • 219
  • 2
  • 11
  • What do you think running as a daemon has to do with it? If you run the script without using `Proc::Daemon`, does it use less than 100% CPU? – mob Jan 21 '11 at 20:02
  • If not running as a daemon, there won't be continuous polling. As a standalone script it will just read the elements of the queue (which in a given state won't usually be too many), perform the operations and exit. – Mohit Chawla Jan 21 '11 at 20:07
  • Also, on IRC I was suggested to look into event based programming using reflex/AnyEvent etc. – Mohit Chawla Jan 21 '11 at 20:08

3 Answers3

3

I bet it is your for loop and checking for new queue entries.

There are ways to watch a directory for file changes. These ways are OS dependent but there might be a Perl module that wraps them up for you. Use that instead of busy looping. Even with a sleep delay, the looping is inefficient when you can have your program told exactly when to wake up by an OS event.

File::ChangeNotify looks promising.

daxim
  • 39,270
  • 4
  • 65
  • 132
Zan Lynx
  • 53,022
  • 10
  • 79
  • 131
1

Maybe you don't want truly continuous polling. Is keep-checking-the-queue-for-new-entries a CPU-intensive part of the code, even when the queue is empty? That would explain why your processor is always busy.

Try putting a sleep 1 statement at the very top (or very bottom) of the while loop to let the processor rest between queue checks. If that doesn't degrade the program performance too much (i.e., if everyone can tolerate waiting an extra second before the company calendars get updated) and if the CPU usage still seems high, try sleep 2, sleep 5, etc.

mob
  • 117,087
  • 18
  • 149
  • 283
  • For now, that does the trick on test servers! Will see if there's a significant difference in production servers. Thanks again. – Mohit Chawla Jan 22 '11 at 06:46
1
cpan Linux::Inotify2

The kernel knows when files change and sends this information to your program which runs the sub. Maybe this will be better because the program will run the sub only when the file is changed.

Dominik Honnef
  • 17,937
  • 7
  • 41
  • 43
coalabear
  • 11
  • 1