7

I am implementing an algorithm and want to measure the time and memory consumption of it. To my aid I've written my own measuring utility, which reads from /proc/[pid]/stat the amount of user and system time consumed, and also the virtual memory and resident set peak sizes. (I am not 100% clear on what the difference is between these two memory statistics, but that's not the question at hand.)

So far so good, but along comes a third-party implementation against which I wish to compare my own programs. As I do not want to fiddle with its source, I am unable to use my own measuring program to collect data on its efficiency (I might be able to, but it would require me to rethink my measuring scheme). So I find /usr/bin/time also takes these measures.

As I compare the output, I find /usr/bin/time does report the same time usage as my own tool, but memory data is much different. First of all, /usr/bin/time does not report anything on virtual memory usage, it only provides a post on max resident set size. Second, the resident set size reported by /usr/bin/time is about six to eight times larger than my own measure.

So, I've been trying to find out how /usr/bin/time actually performs its measuring, and why the difference is so fundamental. Also, which is the correct value?

Example of output (units are in MB):

<program>: <virtual mem> <resident set size>
autotest (my own utility): 23.2266 2.19531
/usr/bin/time: N/A 11.23437

I am using GNU/Linux 3.8.13 Mageia 3 x86_64.

EDIT: As a confidence boost, I find that the KDE System Monitor supports the data that my own utility reports. In fact, it gets its information from the same place as I do. So trusting data from /proc/[pid]/stat should be pretty safe. But the question remains...

EDIT2: From the help of the answers below, it's deduced that wait3() is the command used by GNU time. It returns data in the form described in man page getrusage(2). On RSS, it says:

This is the maximum resident set size used (in kilobytes).

The man page also refers to proc/[pid]/stat, which says

Resident Set Size: number of pages the process has in real memory. This is just the pages which count toward text, data, or stack space. This does not include pages which have not been demand-loaded in, or which are swapped out.

So, is the second version more accurate in just measuring my program's memory usage, and the first version also measures some kind of external-library-usage?

Mats_SX
  • 175
  • 2
  • 9
  • 2
    Why not read the source code for it? – crashmstr Oct 21 '13 at 12:29
  • Have you looked into this answer yet: http://stackoverflow.com/questions/774556/peak-memory-usage-of-a-linux-unix-process – Paul Oct 21 '13 at 12:43
  • (Prev comment was incorrect.) Oh. Duh. Didn't think of that. Doing so showed that it performs `wait3()`. – Mats_SX Oct 21 '13 at 12:46
  • @Paul: Yes, it doesn't seem to measure overall peak usage, which is really what I want to know. – Mats_SX Oct 21 '13 at 13:02
  • Define what you mean by memory usage. The problem is that there are several different possible definitions: do you count memory in a `mmap`ped file, for example. Address space? Total amount swapped in at any point during execution? Maximum amount swapped in at any given time? Total program image, including data which was never swapped in? – James Kanze Oct 21 '13 at 13:07
  • @JamesKanze, I don't understand the difference between these things. Isn't there some consensus on what "memory usage" is? From an algorithmic POV, it's non-ambiguous what "space" refers to, and this is what I want to quantify. – Mats_SX Oct 21 '13 at 13:37
  • From an algorithmic POV, space is address space. I'm not sure that's what you're interested in; with virtual memory, the address space is only vaguely related to actual memory. The address space occupied by various error handling routines, for example, may never be loaded into actual memory (unless the error occurs). – James Kanze Oct 21 '13 at 16:55
  • But aren't these things a constant overhead? If so, that's not a problem for me. I am mostly interested in the difference in memory used as I "increase" my input size, to see whether this increase is large or small (ultimately, in terms of complexity). – Mats_SX Oct 22 '13 at 07:52
  • At what point did you measure the RSS? getrusage, as the manual says, gives you the peak value, not the current value. Your tool and KDE's tool is probably giving you the current value. – syockit Mar 08 '23 at 00:38

2 Answers2

3

I didn't check source of time. But There are wait3 and wait4 functions for getting rusage of child process:

pid_t wait3(int *status, int options, struct rusage *rusage);
pid_t wait4(pid_t pid, int *status, int options, struct rusage *rusage);

struct rusage contains the maximum resident set size value.

Man page getrusage(2) describes the struct rusage.

SKi
  • 8,007
  • 2
  • 26
  • 57
2

I looked at the source (it's very short). It uses getrusage().

Timmmm
  • 88,195
  • 71
  • 364
  • 509