Detect process exit on OSX

Question

I want to be notified right before a process exits on OSX so that I can gather statistics about the process before it dies. (Example specifics: I want to aggregate CPU usage for processes that have many child processes that spawn and die quickly, but use large amounts of CPU. When sampling CPU usage via things like proc_pidinfo, processes that are born and die at rates similar to my sampling rate do not get adequately captured in my aggregate statistics. I would like to be notified when processes die so that I can sum up their total user and system time.

So far, the approach that has seemed like it will work the best has been to use libdtrace, but I can't figure out how to set a probe for process exit using the dtrace command, much less set up libdtrace from a C program. Any tips on setting such a dtrace probe as well as tutorials on how to use libdtrace would be greatly appreciated.

EDIT:

Alright, following the advice of a few commenters, I have managed to create a program that uses libdtrace and reliably triggers off of a process' exit. This is great, but unfortunately I can't seem to get the PID of the process that is exiting. It seems that calling printf() in my D program and trying to format integers is badly broken. Specifically, if I run this program which is supposed to print out the name and PID of a process right as it's exiting, it fails miserably. It prints some other integer, and indeed that integer is printed no matter what I try to output. If I change the D program from

syscall::*exit*:entry {
    printf(\"%s %d\\n\", execname, curpsinfo->pr_pid);
};

To just

syscall::*exit*:entry {
    printf(\"%d\\n\", 100);
};

It just prints out the first three digits of the mystery integer. Note that the process names are correct, it's just the integer -> string conversion that is failing, I believe. Running the main dtrace program with the above D programs works properly, but I want to integrate this into a C program that I have already written a lot into, and piping subcommand outputs into that program isn't really the way I want to move forward.

Help on how to get either the buffered output from libdtrace to work properly, or alternatively to get the PID as an integer rather than a string would be great.

Answer 1

There is an Apple technical note for this subject.

TN2050 Observing Process Lifetimes Without Polling

For monitoring an arbitrary process, this guide suggests kqueues.

We can obtain PIDs of running processes by this way [unable to detect application running with another user (via switch user) or this way [Programmatically check if a process is running on Mac.

Listing 8 Using kqueues to monitor a specific process

static pid_t gTargetPID = -1;
    // We assume that some other code sets up gTargetPID.

- (IBAction)testNoteExit:(id)sender
{
    FILE *                  f;
    int                     kq;
    struct kevent           changes;
    CFFileDescriptorContext context = { 0, self, NULL, NULL, NULL };
    CFRunLoopSourceRef      rls;

    // Create the kqueue and set it up to watch for SIGCHLD. Use the 
    // new-in-10.5 EV_RECEIPT flag to ensure that we get what we expect.

    kq = kqueue();

    EV_SET(&changes, gTargetPID, EVFILT_PROC, EV_ADD | EV_RECEIPT, NOTE_EXIT, 0, NULL);
    (void) kevent(kq, &changes, 1, &changes, 1, NULL);

    // Wrap the kqueue in a CFFileDescriptor (new in Mac OS X 10.5!). Then 
    // create a run-loop source from the CFFileDescriptor and add that to the 
    // runloop.

    noteExitKQueueRef = CFFileDescriptorCreate(NULL, kq, true, NoteExitKQueueCallback, &context);
    rls = CFFileDescriptorCreateRunLoopSource(NULL, noteExitKQueueRef, 0);
    CFRunLoopAddSource(CFRunLoopGetCurrent(), rls, kCFRunLoopDefaultMode);
    CFRelease(rls);

    CFFileDescriptorEnableCallBacks(noteExitKQueueRef, kCFFileDescriptorReadCallBack);

    // Execution continues in NoteExitKQueueCallback, below.
}

static void NoteExitKQueueCallback(
    CFFileDescriptorRef f, 
    CFOptionFlags       callBackTypes, 
    void *              info
)
{
    struct kevent   event;

    (void) kevent( CFFileDescriptorGetNativeDescriptor(f), NULL, 0, &event, 1, NULL);

    NSLog(@"terminated %d", (int) (pid_t) event.ident);

    // You've been notified!
}

Answer 2

The following example takes a UNIX process ID as argument, and watches up to 20 seconds, and reports if the process terminates in that time

// cc test.c -framework CoreFoundation -O
#include <CoreFoundation/CoreFoundation.h>
#include <unistd.h>
#include <sys/event.h>
static void noteProcDeath(CFFileDescriptorRef fdref, CFOptionFlags callBackTypes, void *info) {
    struct kevent kev;
    int fd = CFFileDescriptorGetNativeDescriptor(fdref);
    kevent(fd, NULL, 0, &kev, 1, NULL);
    // take action on death of process here
    printf("process with pid '%u' died\n", (unsigned int)kev.ident);
    CFFileDescriptorInvalidate(fdref);
    CFRelease(fdref); // the CFFileDescriptorRef is no longer of any use in this example
}
// one argument, an integer pid to watch, required
int main(int argc, char *argv[]) {
    if (argc < 2) exit(1);
    int fd = kqueue();
    struct kevent kev;
    EV_SET(&kev, atoi(argv[1]), EVFILT_PROC, EV_ADD|EV_ENABLE, NOTE_EXIT, 0, NULL);
    kevent(fd, &kev, 1, NULL, 0, NULL);
    CFFileDescriptorRef fdref = CFFileDescriptorCreate(kCFAllocatorDefault, fd, true, noteProcDeath, NULL);
    CFFileDescriptorEnableCallBacks(fdref, kCFFileDescriptorReadCallBack);
    CFRunLoopSourceRef source = CFFileDescriptorCreateRunLoopSource(kCFAllocatorDefault, fdref, 0);
    CFRunLoopAddSource(CFRunLoopGetMain(), source, kCFRunLoopDefaultMode);
    CFRelease(source);
    // run the run loop for 20 seconds
    CFRunLoopRunInMode(kCFRunLoopDefaultMode, 20.0, false);
    return 0;
}

For those who barely know C:
build: cc test.c -framework CoreFoundation -O
run: ./a.out 57168
57168 is the pid of the process being monitored. Kill it to test!

Surely you can increase the 20 seconds to make it last as long as you want.