10

I am working on an app that monitors network usage. However I noticed many ways to do this does not allow exclusion of local traffic (say, Time Machine).

I am looking for a way to exclude local traffic, and only monitors usage that goes directly to/from the internet.

Update: Thank you for your replies, now I know how to find if the traffic is local, but I still don't know how I can calculate total in/out bytes (sorry if I didn't elaborate earlier). I have no way of knowing how many bytes are sent/received locally (or to the internet) in a certain period of time, or since the OS starts. This problem is further complicated by the fact processes are launched or killed when the OS is running.

The answer to the question How to get network adapter stats in linux/Mac OSX? gives an interesting way of summing up total usage but it doesn't help because the usage it sums up are interface statistics.

Update 2: I've posted my final solution to this. Please scroll down a bit to see.

Community
  • 1
  • 1
koo
  • 2,888
  • 1
  • 23
  • 29
  • Look at traffic on interfaces other than lo. Or the one that the default route points to. Not sure how to do this with Mac OS X-specific calls (or I'd post this as an answer), but the Unix tools can do it... – derobert Feb 03 '11 at 17:38
  • Excluding every local interface's subnet should do a reasonable job. `getifaddrs(3)` should do fine; the only Mac-specific stuff you might want is to watch for interfaces to come up/down. – Nicholas Riley Feb 19 '11 at 07:06
  • @derobert Thank you, but aren't other interfaces supposed to have local traffic as well? – koo Apr 01 '11 at 04:53
  • I've answered you comment in a answer instead of another comment, because my answer is *way* too long for a comment. – derobert Apr 01 '11 at 18:09

8 Answers8

2

you need to read the source for ifconfig(8), which describes how to get the status of every attached network interface.

pay particular attention to in_status(), which gets the inet address and netmask of an interface.

when the source or destination address in the traffic has the same host as a local interface

int is_local =
(src && netmask) == (ifaddr && netmask)
|| (dst && netmask) == (ifaddr && netmask)

then you can be sure that it is local

http://www.opensource.apple.com/source/network_cmds/network_cmds-307/ifconfig.tproj/ifconfig.c

rbp
  • 1,850
  • 15
  • 28
2

Answering you comment about which interfaces carry local traffic is actually complicated, because it depends on what you mean by local traffic.

What “Local” Means

The easiest meaning of "local traffic" is traffic that does not leave the machine its generated on (two programs on the same machine talking to each other, for example). This traffic all goes over lo. This is one thing that people mean when they say local (and what I was thinking of when I answered).

The next easiest meaning would be "IP traffic destined to machines on the same subnet". That'd be traffic that has a destination address inside the local subnet. The easiest way to count this is going to be either the routing table (if Mac OS X counts traffic stats per route, the routes on the various gateways will give you non-local traffic) or with a firewall rule. This probably isn't want anyone means when they say "local traffic".

Another meaning would be "IP traffic destined to machines in this (physical) location". E.g., at my office we have several subnets in use, with routers between them, but traffic from one subnet to the other is still clearly local. You need network knowledge to distinguish local from non-local traffic with this definition.

Another meaning would be "IP traffic destined to machines in my organization". This is a reasonable meaning depending on how your network is set up (e.g., maybe you have fast fiber between your locations, but your Internet connections are much slower, or charged per-GB). Requires in-depth knowledge of the network to figure if a destination is going to be local or not—and, with things like VPNs, that may vary over time.

Finally, "Internet traffic" isn't the opposite of any of those. Sometimes, for example, what appears to be a local machine on your Ethernet segment is actually over a VPN, over the Internet (this isn't crazy, it's very useful for when remote users need to use various Windows services). Traffic inside your organization can easily travel over an Internet VPN.

Cheating in Simple Networks

If the network is very simple, with there being only one internal subnet, only one router, and all traffic not to that internal subnet being Internet traffic, you can cheat and solve this. This probably applies to the vast majority of home networks, and many small business ones as well.

Using firewall rules

In a simple network setup, you can probably make some assumptions, and get a close enough answer by counting traffic as non-local if:

  • the destination MAC address is the default gateway's MAC address; and
  • the destination IP address is not the default gateway's IP address

alternatively:

  • the destination IP address is not within the subnet of the network interface the default route goes out

You can probably create a firewall rule to count either of those. At least with Linux iptables you can, and I'm pretty sure BSD pf, and probably Mac OS X.

Alternate Approach: SNMP

Finally, if you can't use a firewall rule (as that'd require root), you could hope that the default gateway responds to SNMP community public, explore all its interfaces, and find the one with a off-subnet IP address, and then assume that is the Internet link. Then you can ask the router for traffic counts on that interface.

Of course, you'll find that many SOHO routers don't support SNMP, and those that do probably don't have it turned on.

derobert
  • 49,731
  • 15
  • 94
  • 124
  • Thanks! That's a very good answer. But I think I should just give up. I checked `ipsw` (IP firewall and traffic shaper control program) from BSD system manager's manpage, but it can't be used to give useful statistics. And many existing linux solutions cannot be ported because they make use of `/proc/net`. Also SNMP is not what I really looking for because my app is not oriented to the professionals alike. – koo Apr 01 '11 at 19:58
1

The best way is to find the 'external' ip address through the eth0, eth1, or whatever adapter with a system call to ifconfig. Then pull logs for whatever system (messages, syslog, whatever) and write a filter for that external ip address. To make it nicer and more portable, write a regex that will filter for publicly routable IPs only and just filter messages log for that 'external' ip address.

kirtcathey
  • 343
  • 1
  • 6
  • 16
0

I think, an approximate solution: getifaddrs can be used to get statistics on network usage.

It can get separate statistics for Wi-Fi and WWAN interfaces.

You might find more information from :

http://www.gsp.com/cgi-bin/man.cgi?section=3&topic=getifaddrs

iHS
  • 5,372
  • 4
  • 31
  • 49
  • Thank you! I looked at getifaddrs. But it is not the thing I wanted. What I need is a way to separate local & internet traffic statistics which getifaddrs does not provide. To make things worse, the data structure of the statistics for AF_LINK families (->ifa_data->ifi_obytes and ifi_ibytes) are 32-bit unsigned integers, which means they overflow after statistics exceed approximately 4 GB. – koo Apr 01 '11 at 18:08
0

It depends on how you define "local", but a common definition would be to look at the network mask.

For example, if your IP (ie the IP of the interface you monitor is

10.33.52.123
netmask 255.255.255.0

that would mean every IP-packet with both source-IP and destination-IP 10.33.52.xx is local.

I don't know cocoa or objective-c, but you can probably use some of these functions helping you extract the network from an IP-address: http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man3/inet_network.3.html

Rolf Rander
  • 3,221
  • 20
  • 21
0

Don't know how to implement it in objective-c but the idea is that you get the address of the network you are in (you can figure this out from network class(A,B,C) based from your local ip or from bits in netmask if it's not standard), then just check the outgoing connection's address. If the destination is not in your local network, calculate traffic; if it's in, just do nothing.

Michael Myers
  • 188,989
  • 46
  • 291
  • 292
Viktor
  • 702
  • 3
  • 8
  • 20
0

There are three ranges of non-routable IP addresses, and they are commonly used as the address ranges for NAT services. Any address that is not in one of the non-routable address ranges is an external address.

Of course if you are not behind a NAT router, the task is harder (and technically all the addresses short of 127.0.0.1 are external at this point).

The non-routable IP ranges are:

10.0.0.0 - 10.255.255.255

172.16.0.0 - 172.31.255.255

192.168.0.0 - 192.168.255.255

David Brown
  • 2,727
  • 1
  • 16
  • 5
  • This isn't true; internal addresses don't have to be RFC1918 space. There are plenty of times you'll use non-1918 space for machines on a network, e.g., servers. Traffic from one server to another on the same ethernet segment is still local, and is definitely not Internet traffic. – derobert Apr 01 '11 at 18:08
0

The final working solution I have is to use libpcap to achieve this. Of course there are some downsides, which includes it requires elevated privileges and must capture all filtered packets to calculate statistics, but at least it works perfectly well.

Many documentations and tutorials on libpcap is fairly thorough and clear, I suggest every one interested in this solution to look at those with relatively little google-fu effort.

Also it may interest a few that my filter for internet traffic is simply the following -

- (NSString *)_internetFilterStringForInterface:(AKNetworkInterface *)interface
    inOrOut:(BOOL)inYesOutNo
{
    if (![interface net] || ![interface mask] || IsEmpty([interface addresses]))
    {
        return nil;
    }

    NSString *hostType = inYesOutNo ? @"dst" : @"src";
    NSString *host = nil;
    for (NSString *hostComponent in [interface addresses])
    {
        if (IsEmpty(hostComponent)) continue;
        if (!host)
            host = [NSString stringWithFormat:@"(%@ host %@", hostType, hostComponent];
        else
            host = [host stringByAppendingFormat:@" or %@ host %@", hostType, hostComponent];
    }
    host = [host stringByAppendingString:@")"];

    NSString *net = [interface netString];
    net = [net stringByReplacingOccurrencesOfString:@".0" withString:@""];

    NSString *filter = [NSString stringWithFormat:
                        @"ip and (not %@ net %@) and %@",
                        inYesOutNo ? @"src" : @"dst",
                        net, host];
    return filter;
}

The filter is designed with some of the answers about what counts as 'local traffic', I know it does not encompass some edge cases such as double NAT configurations, etc., but I would like to see suggestions about this.

I know net = [net stringByReplacingOccurrencesOfString:@".0" withString:@""]; is just a quick hack which could easily fail under some peculiar circumstances but hey no one is complaining, at least not yet.

koo
  • 2,888
  • 1
  • 23
  • 29