Can an interrupt be programmed like this?

Question

I am creating a shell which can spawn programs in the background. When a program starts to use a certain amount of memory, like 100MB, I want an interrupt to be triggered that will cause a function to run. The alternative is to have a process keep running and checking the status of background processes. Using an interrupt seems to use less CPU resources. Is that correct? Is it possible to do this?

I'm trying to do this in Linux.

Answer 1

In Linux, you can set a resource limit for a process with setrlimit. In your case, RLIMIT_AS or RLIMIT_DATA can be used. malloc, which uses brk, will fail on reaching the limit. You can monitor the value returned by malloc, and raise a signal yourself.

Answer 2

One way to do this on Linux would be to use ptrace. You'd then use PTRACE_PEEKUSER to look at the values the child is supplying to the kernel when it makes system calls. You'd be looking for things like mmap and/or brk. The function being called is defined by the value in EAX when int 80 is executed (sorry, offhand I don't know the numbers for each mmap or brk).

Answer 3

When a program starts to use a certain amount of memory, like 100MB, I want an interrupt to be triggered that will cause a function to run.

Depending on requirements. If capturing malloc calls is enought you might be interested:

• Tutorial: Function Interposition in Linux , an example how to make you own malloc substitute, here another relared tutorial : Inside memory management
• Create a wrapper function for malloc and free in C - stackoverflow.com , especially malloc hooks
• mtrace at GNU C Library.

If, you'd like to capture brk(2) as well - (related article) , you might like to follow in some of tracing solution

• ptrace()
• pinktrace

or binary instrumentation

• Valgrind