Double Progress Bar in Python

Question

Is there a way to create a double progress bar in Python? I want to run two loops inside each other. For each loop I want to have a progress bar. My program looks like:

import time
for i1 in range(5):
    for i2 in range(300):
        # do something, e.g. sleep
        time.sleep(0.01)
        # update upper progress bar
    # update lower progress bar

The output somewhere in the middle should look something like

50%|############################                                  |ETA: 0:00:02
80%|##################################################            |ETA: 0:00:04

The already existing really cool progressbar module doesn't seem to support that.

Answer 1

Use the nested progress bars feature of tqdm, an extremely low overhead, very customisable progress bar library:

$ pip install -U tqdm

Then:

from tqdm import tqdm
# from tqdm.auto import tqdm  # notebook compatible
import time
for i1 in tqdm(range(5)):
    for i2 in tqdm(range(300), leave=False):
        # do something, e.g. sleep
        time.sleep(0.01)

(The leave=False is optional - needed to discard the nested bars upon completion.)

You can also use from tqdm import trange and then replace tqdm(range(...)) with trange(...). You can also get it working in a notebook.

Alternatively if you want just one bar to monitor everything, you can use tqdm's version of itertools.product:

from tqdm.contrib import itertools
import time
for i1, i2 in itertools.product(range(5), range(300)):
    # do something, e.g. sleep
    time.sleep(0.01)

Answer 2

I basically just want to add to the answer of @casper.dcl. In the slightly different case, where you have two nested for loops and want just a SINGLE progress bar you can do the following.

from tqdm import tqdm
import time
n = 5
m = 300
with tqdm(total=n * m) as pbar:
    for i1 in tqdm(range(n)):
        for i2 in tqdm(range(m)):
            # do something, e.g. sleep
            time.sleep(0.01)
            pbar.update(1)

I know that was not the question, but it might be still helpful for some folks.

Answer 3

It would require you to move the cursor position. I have written you a hacky thing to do it.

This script relies on the fact that the progressbar module assumes that you are on a fresh line to draw the progress bar. By simply moving the cursor up (using the escape code for "move cursor 1 row up"), and down (just using a newline. I could also use an escape code, but newline is easier and faster), one can maintain multiple progress bars.

import progressbar, time, sys

def up():
    # My terminal breaks if we don't flush after the escape-code
    sys.stdout.write('\x1b[1A')
    sys.stdout.flush()

def down():
    # I could use '\x1b[1B' here, but newline is faster and easier
    sys.stdout.write('\n')
    sys.stdout.flush()

# Total bar is at the bottom. Move down to draw it
down()
total = progressbar.ProgressBar(maxval=50)
total.start()

for i in range(1,51):
    # Move back up to prepare for sub-bar
    up()

    # I make a new sub-bar for every iteration, thinking it could be things
    # like "File progress", with total being total file progress.
    sub = progressbar.ProgressBar(maxval=50)
    sub.start()
    for y in range(51):
        sub.update(y)
        time.sleep(0.005)
    sub.finish()

    # Update total - The sub-bar printed a newline on finish, so we already
    # have focus on it
    total.update(i)
total.finish()

This is of course a bit hacky, but it gets the job done. I hope that it is useful.

Answer 4

Once here was an answer by @yurenchen (which was deleted), that advertised rich library, it has progress bar routines described here in docs.

Note. See also my other two answers in this thread about progress bar - using enlighten lib and using ASCII art.

Rich library can be installed by python -m pip install rich.

Minimal example that shows stack of three progress bars of different colors is:

Try it online!

import time

from rich.progress import Progress

with Progress() as progress:

    task1 = progress.add_task("[red]Downloading...", total=1000)
    task2 = progress.add_task("[green]Processing...", total=1000)
    task3 = progress.add_task("[cyan]Cooking...", total=1000)

    while not progress.finished:
        progress.update(task1, advance=0.5)
        progress.update(task2, advance=0.3)
        progress.update(task3, advance=0.9)
        time.sleep(0.02)

which produces following colorful console output (+ aciinema link):

Answer 5

Use enlighten:

import time
import enlighten

manager = enlighten.get_manager()
ticks = manager.counter(total=100, desc="Ticks", unit="ticks", color="red")
tocks = manager.counter(total=20, desc="Tocks", unit="tocks", color="blue")

for num in range(100):
    time.sleep(0.1)  # Simulate work
    print("The quick brown fox jumps over the lazy dog. {}".format(num))
    ticks.update()
    if not num % 5:
        tocks.update()

manager.stop()

Answer 6

A little late in the game, but here's an answer using nothing but tqdm

import re
from time import sleep
from tqdm import trange

class DescStr:
    def __init__(self):
        self._desc = ''

    def write(self, instr):
        self._desc += re.sub('\n|\x1b.*|\r', '', instr)

    def read(self):
        ret = self._desc
        self._desc = ''
        return ret

    def flush(self):
        pass


rng_a = trange(10)
desc = DescStr()
for x in rng_a:
    for y in trange(10, file=desc, desc="Y"):
        rng_a.set_description(desc.read())
        sleep(0.1)

which yields:

Y:  90%|######### | 9/10 [00:00<00:00,  9.55it/s]: 100%|##########| 10/10 [00:10<00:00,  

Answer 7

Inspired by this answer, I also tried enlighten python library and wrote my simple helper function pit() for wrapping iterators inside for-loop (top of code) and provided example of usage (bottom of code) plus live terminal-screencast.

Note. See also my other two answers in this thread about progress bar - using rich lib and using ASCII art.

Main difference to linked answer is that pit() allows to be used inside for-loop to wrap iterator, instead of using manual .update() method, this iterator-wrapping functionality is lacking in englighten, that's why I decided to implement my own.

As one can see in Accepted answer other famous progress bar libraries like tqdm already have this functionality of wrapping iterators in for-loop and also multiple progress bars in nested loops.

Works in color both in Linux and Windows.

Try it online!

# Helper Progress Iterator
# Needs: python -m pip install enlighten

def pit(it, *pargs, **nargs):
    import enlighten
    global __pit_man__
    try:
        __pit_man__
    except NameError:
        __pit_man__ = enlighten.get_manager()
    man = __pit_man__
    try:
        it_len = len(it)
    except:
        it_len = None
    try:
        ctr = None
        for i, e in enumerate(it):
            if i == 0:
                ctr = man.counter(*pargs, **{**dict(leave = False, total = it_len), **nargs})
            yield e
            ctr.update()
    finally:
        if ctr is not None:
            ctr.close()


####### Usage Example ########

import time

def Generator(n):
    for i in range(n):
        yield i

for i in pit(range(2), color = 'red'):
    for j in pit(range(3), color = 'green'):
        for k in pit(Generator(4), total = 4, color = 'blue'):
            for l in pit(Generator(5)):
                print(i, j, k, l)
                time.sleep(0.05)

Output (+ ascii-video):

Answer 8

I sketched up a standalone, simple progressbar for Python3.6+. No tqdm, no other dependencies, no hacks.

def myprogress(current, whole=1, n=30, bars=u'▕▏▎▍▌▋▊▉', full='▉', empty='▕'): 
    """ current and whole can be an element of a list being iterated, or just two numbers """
    p = (whole.index(current))/len(whole)+1e-9 if type(whole)==list else current/whole+1e-9 
    return f"{full*int(p*n)}{bars[int(len(bars)*((p*n)%1))]}{empty*int((1-p)*n)} {p*100:04.1f}%" 

In pure Python it is hard to re-write more than the last line. But you can just stack two bars next to each other. (A single-line solution also nicely works e.g. with window decoration in a GUI!)

for x in range(300):  ## accepting numerical value
     print(myprogress(x/300), ' '*5, myprogress(x/321), end='\r')
     for busyloop in range(10**5): pass

It computes the progress either as ratio of two numbers, or finds an element in a list being iterated. (If you iterate a numpy.array, it is easy to convert it to a list.) So this is also possible:

l = ['apples', 'bananas', 'cherries', 'durians']  ## accepting an element from list being iterated
for x in l: 
     print(myprogress(x, whole=l), ' '*5, myprogress(x, whole=l), end='\r')
     for busyloop in range(10**7): pass

In the first example you get:

▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▎▕▕▕▕▕▕▕▕ 71.0%      ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▕▕▕▕▕▕▕▕▕▕ 66.4%

                                                                                                    

It's so simple it must be public domain.

PS: if you like Pacman, you may revert the progress right to left and use: bars='ᗦᗠᗦᗠᗦᗠ'

Answer 9

Here is a simple method that displays progress for an outer and an inner loop:

from tqdm import tqdm
from time import sleep

pbar = tqdm(range(10))
for i in pbar:
    for j in range(20):
        pbar.set_postfix({'inner': j})
        sleep(.2)

It's not exactly what you asked for: The inner loop here is only displayed as incrementing numbers, while the progress bar shows the outer loop progress. But it's a useful visualization of nested loops.

Here is a snapshot:

 30%|███       | 3/10 [00:14<00:33,  4.77s/it, inner=12]

The "inner" counter increments constantly as the progress bar for the outer loop advances slowly.

Update:

You can combine this solution with dominecf's solution. The following uses tqdm for the outer loop and integrates an inner loop using dominecf's function (with minor modifications):

import tqdm
import time

def myprogress(curr, N, width=10, bars = u'▉▊▋▌▍▎▏ '[::-1],
               full='█', empty=' '): 
    p = curr / N 
    nfull = int(p * width)
    return "{:>3.0%} |{}{}{}| {:>2}/{}"\
        .format(p, full * nfull,
                bars[int(len(bars) * ((p * width) % 1))],
                empty * (width - nfull - 1),
                curr, N)


pbar = tqdm.tqdm(range(10),
                 bar_format='{l_bar}{bar:10}{r_bar}{bar:-10b}')
for i in pbar:
    for j in range(20):
        pbar.set_postfix_str(myprogress(j, 20))
        time.sleep(.2)

Here is a snapshot:

 30%|███       | 3/10 [00:14<00:34,  4.90s/it, 60% |██████    | 12/20]                                                                                                                                      

Answer 10

This can be easily done with atpbar.

For example:

import time, random
from atpbar import atpbar

for i in atpbar(range(4), name='outer'):
    n = random.randint(1000, 10000)
    for j in atpbar(range(n), name='inner {}'.format(i)):
        time.sleep(0.0001)

The code above has nested for loops. The outer loop iterates four times. For each iteration of the outer loop, the inner loop iterates the number of times that is randomly selected. The progress bar for the inner loop moves up as the loop completes. The active progress bars stay at the bottom. A snapshot of progress bars might look like

 100.00% :::::::::::::::::::::::::::::::::::::::: |     3287 /     3287 |:  inner 0
 100.00% :::::::::::::::::::::::::::::::::::::::: |     5850 /     5850 |:  inner 1
  50.00% ::::::::::::::::::::                     |        2 /        4 |:  outer  
  34.42% :::::::::::::                            |     1559 /     4529 |:  inner 2

Answer 11

Inspired by simplicity of answer of @dominecf, just for fun I implemented a helper wrapper function pbiter() that can be used in loops to show progress for any iterables. pbiter() uses @dominecf's implementation of myprogress().

Note. See also my other two answers in this thread about progress bar - using enlighten and using rich lib.

Don't judge this answer too much, it is only for hackery fun of implementing progress from scratch in pure Python, this answer is not meant to be used in any production environment, use tqdm or enlighten modules in real application for doing progress.

See my other answer to same Question, that answer shows how to use enlighten module for progress.

pbiter() from this answer can be very simply used with any iterables in nested loops like following:

for a in pbiter(range(12)):
    for b in pbiter(generator_nums(13)):
        for c in pbiter(generator_nums(7), total = 7):
            time.sleep(0.03)

Progress bar total length is figured out either by len(it) if it is available for iterable (e.g. for range(start, stop, step) it is always available), or by providing total = ... param, otherwise progress decays exponentially with multiplier 0.1 (which shows a nice approximation). In three example nested loops above second loop has this exponential behaviour.

Full code below. See ascii-video located after code.

Try it online!

def myprogress(current, whole=1, n=30, bars=u'▕▏▎▍▌▋▊▉', full='▉', empty='▕'): 
    """ current and whole can be an element of a list being iterated, or just two numbers """
    p = (whole.index(current))/len(whole)+1e-9 if type(whole)==list else current/whole+1e-9 
    return f"{full*int(p*n)}{bars[int(len(bars)*((p*n)%1))]}{empty*int((1-p)*n)} {p*100:>6.2f}%" 

def pbiter(it, *, total = None, width = 36, _cfg = {'idx': -1, 'pbs': {}, 'lline': 0}):
    try:
        total = total or len(it)
    except:
        total = None
    
    _cfg['idx'] += 1
    idx = _cfg['idx']
    pbs = _cfg['pbs']
    pbs[idx] = [0, total, 0]
    
    def Show():
        line2 = ' '.join([
            myprogress(e[1][0], max(e[1][0], e[1][1] or
                max(1, e[1][0]) / max(.1, e[1][2])), width // len(pbs))
            for e in sorted(pbs.items(), key = lambda e: e[0])
        ])
        line = line2 + ' ' * (max(0, _cfg['lline'] - len(line2)) + 0)
        print(line, end = '\r', flush = True)
        _cfg['lline'] = len(line2)
    
    try:
        Show()
        for e in it:
            yield e
            pbs[idx][0] += 1
            pbs[idx][2] += (1. - pbs[idx][2]) * .1
            Show()
        pbs[idx][2] = 1.
        Show()
    finally:
        del pbs[idx]

def test():
    import time

    def generator_nums(cnt):
        for i in range(cnt):
            yield i

    for a in pbiter(range(12)):
        for b in pbiter(generator_nums(13)):
            for c in pbiter(generator_nums(7), total = 7):
                time.sleep(0.03)

test()

ASCII-video output (see also asciinema video page):

If for some reason you don't have loops and still want to use my pbiter(), then you can use it through regular built-in next() operation, as following:

# Create 3 progress bars, they are at 0% point now
a = pbiter(range(5))
b = pbiter(range(4))
c = pbiter(range(3))
# Some lines of code later, advance progress "a"
next(a)
# And later ...
next(b)
# And later ...
next(b)
# Later ...
next(a); next(c)
# Later ...
next(c); next(b)

in other words you can create and advance progress bars manually in any order and at any place of code.