How to count how many data has been downloaded in the last second? (FTP)

Question

I want to know how many data has been downloaded in the last 1 second.
I don't have a code yet but I was wondering when I should start counting this 1 second and how to do it.
Should I start counting before retrbinary() or after? Or am I totally wrong?

Answer 1

First, there are ready-made implementations for transfer progress display, including the transfer speed.

For an example, the progressbar2 module. See Show FTP download progress in Python (ProgressBar).

The progressbar2 by default displays FileTransferSpeed widget, what is an average transfer speed since the download started.

Though note that speed displays usually do not show such speed. They display an average speed over last few seconds. That makes the value more informative. The progressbar2 has AdaptiveTransferSpeed widget for that. But it seems to be broken.

---

If you want to implement the calculation on your own, and are happy with the simple average transfer speed since the download started, it is easy:

from ftplib import FTP
import time
import sys
import datetime

ftp = FTP(host, user, passwd)

print("Downloading")
total_length = 0
start_time = datetime.datetime.now()

def write(data):
   f.write(data)
   global total_length
   global start_time
   total_length += sys.getsizeof(data)
   elapsed = (datetime.datetime.now() - start_time)
   speed = (total_length / elapsed.total_seconds())
   print("\rElapsed: {0} Speed: {1:.2f} kB/s".format(str(elapsed), speed / 1024), end="")

f = open('file.dat', 'wb')
ftp.retrbinary("RETR /file.dat", write)
f.close()

print()
print("done")    

---

It is a way more difficult to calculate the average speed in the last seconds. You have to remember the amount of data transferred at past moments. Stealing (and fixing) the code from AdaptiveTransferSpeed, you will get something like:

sample_times = []
sample_values = []
INTERVAL = datetime.timedelta(milliseconds=100)
last_update_time = None
samples=datetime.timedelta(seconds=2)
total_length = 0

def write(data):
   f.write(data)

   global total_length

   total_length += sys.getsizeof(data)
   elapsed = (datetime.datetime.now() - start_time)

   if sample_times:
       sample_time = sample_times[-1]
   else:
       sample_time = datetime.datetime.min

   t = datetime.datetime.now()
   if t - sample_time > INTERVAL:
       # Add a sample but limit the size to `num_samples`
       sample_times.append(t)
       sample_values.append(total_length)

       minimum_time = t - samples
       minimum_value = sample_values[-1]
       while (sample_times[2:] and
              minimum_time > sample_times[1] and
              minimum_value > sample_values[1]):
           sample_times.pop(0)
           sample_values.pop(0)

   delta_time = sample_times[-1] - sample_times[0]
   delta_value = sample_values[-1] - sample_values[0]
   if delta_time:
       speed = (delta_value / delta_time.total_seconds())

       print("\rElapsed: {0} Speed: {1:.2f} kB/s".format(
           str(elapsed), speed / 1024), end="")

ftp.retrbinary("RETR /medium.dat", write)