python getting upload/download speeds

Question

I want to monitor on my computer the upload and download speeds. A program called conky already does it with the following in conky conf:

Connection quality: $alignr ${wireless_link_qual_perc wlan0}%
${downspeedgraph wlan0}
DLS:${downspeed wlan0} kb/s $alignr total: ${totaldown wlan0}

and it shows me the speeds in almost real time while I browse. I want to be able to access the same information using python.

Answer 1

You can calculate the speed yourself based on the rx_bytes and tx_bytes for the device and polling those values over an interval

Here is a very simplistic solution I hacked together using Python 3

#!/usr/bin/python3

import time

def get_bytes(t, iface='wlan0'):
    with open('/sys/class/net/' + iface + '/statistics/' + t + '_bytes', 'r') as f:
        data = f.read();
    return int(data)

if __name__ == '__main__':
    (tx_prev, rx_prev) = (0, 0)

    while(True):
        tx = get_bytes('tx')
        rx = get_bytes('rx')

        if tx_prev > 0:
            tx_speed = tx - tx_prev
            print('TX: ', tx_speed, 'bps')

        if rx_prev > 0:
            rx_speed = rx - rx_prev
            print('RX: ', rx_speed, 'bps')

        time.sleep(1)

        tx_prev = tx
        rx_prev = rx

Answer 2

I would look into the psutil module for Python.

Here is a short snippet which prints out the number of bytes sent since you booted your machine:

import psutil
iostat = psutil.net_io_counters(pernic=False)
print iostat[0] #upload only

You could easily expand this to grab the value at a constant interval and diff the two values to determine the number of bytes sent and/or received over that period of time.

Answer 3

In order to have interface specific statistics, the methods already proposed would work just fine.

I'll try instead to suggest a solution for your second request:

It would also be very helpful to know which program was using that bandwidth, but so far I haven't seen anything that can do that.

As already suggested, nethogs prints process specific statistics. To my knowledge, there's no easy way to access these values under /proc and I will therefore explain how nethogs achieves this.

Considering one process with pid PID, nethogs first retrieves a list of all the sockets opened by the process listing the content of /proc/PID/fd:

➜  ~ [1] at 23:59:31 [Sat 15] $ ls -la /proc/21841/fd
total 0
dr-x------ 2 marco marco  0 Nov 15 23:41 .
dr-xr-xr-x 8 marco marco  0 Nov 15 23:41 ..
lrwx------ 1 marco marco 64 Nov 15 23:42 0 -> /dev/pts/15
l-wx------ 1 marco marco 64 Nov 15 23:42 1 -> /dev/null
lrwx------ 1 marco marco 64 Nov 15 23:41 2 -> /dev/pts/15
lrwx------ 1 marco marco 64 Nov 15 23:42 4 -> socket:[177472]

Here we have just one socket and 177472 is the inode number. We will find here all kind of sockets: TCPv4, TCPv6, UDP, netlink. In this case I will consider only TCPv4.

Once all the inode numbers are collected, each socket is assigned an unique identifier, namely (IP_SRC, PORT_SRC, IP_DEST, PORT_DEST). And of course the pairing with the PID is stored as well. The tuple (IP_SRC, PORT_SRC, IP_DEST, PORT_DEST) can be retrieved reading /proc/net/tcp (for TCPv4). In this case:

➜  ~ [1] at 0:06:05 [Sun 16] $ cat /proc/net/tcp | grep 177472
  sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode
  38: 1D00A8C0:1F90 0400A8C0:A093 01 00000000:00000000 00:00000000 00000000  1000        0 177472 1 f6fae080 21 4 0 10 5 

Addresses are expressed as IP:PORT, with IP represented as a 4 bytes LE number. You can then build a key->value structure, where the key is (IP_SRC, PORT_SRC, IP_DEST, PORT_DEST) and value is the PID.

At this point, nethogs captures all the network traffic with libpcap. When it detects a TCP packet it tries to match the tuple (IP_SRC_PACKET, PORT_SRC_PACKET, IP_DEST_PACKET, PORT_DEST_PACKET) against all the connections inside the table. Of course it must try to swap SRC and DEST, the packet could be incoming (DL) or outgoing (UL). If it maches a connection, it retrieves the PID of the process the connection belongs to and it adds the size of the TCP payload to the TX or RX counter. With the number of bytes updated at every packet captured, the transfer speed for each process can be easily calculated.

This, in theory, can be implemented in python with pypcap, even though it needs a bit of work. I have tried to implement something, but it's painfully slow and it requires much more work to be usable. I was monitoring just one PID, with one connection, not updating the connections table, but beyond 3MB/s my script could not keep up with the network traffic.

As you can see it's not exactly trivial. Parsing the output of a tool already available might lead to a better solution and might save you a lot of work.

Answer 4

You could do something dodgy like call conky -i 1 and parse the output:

import subprocess
conky=subprocess.check_output("conky -i 1", shell=True)
lines=conky.splitlines()
print lines[11].split()[1::3]

resulting in:

['1234B', '5678B']

my config looks like:

${scroll 16 $nodename - $sysname $kernel on $machine | }
Uptime: $uptime
Frequency (in MHz): $freq
Frequency (in GHz): $freq_g
RAM Usage: $mem/$memmax - $memperc% ${membar 4}
Swap Usage: $swap/$swapmax - $swapperc% ${swapbar 4}
CPU Usage: $cpu% ${cpubar 4}
Processes: $processes  Running: $running_processes
File systems:
 / ${fs_used /}/${fs_size /} ${fs_bar 6 /}
Networking:
Up: ${upspeed eth0}  - Down: ${downspeed eth0}
Name              PID   CPU%   MEM%
 ${top name 1} ${top pid 1} ${top cpu 1} ${top mem 1}
 ${top name 2} ${top pid 2} ${top cpu 2} ${top mem 2}
 ${top name 3} ${top pid 3} ${top cpu 3} ${top mem 3}
 ${top name 4} ${top pid 4} ${top cpu 4} ${top mem 4}