Why is tqdm printing to a newline instead of updating the same line?

Question

I'm working on a small command-line game in python where I am showing a progress bar using the tqdm module. I listen for user input using the msvcrt module to interrupt the progress. Once interrupted, the user can restart by entering 'restart' into the command line prompt. The second time the progress bar is shown, instead of updating the same line with the progress, it creates a new line each time.

How would I get it to show the progress on the same line?

This code snippet illustrates my use of the progress bar.

def transfer():
    for i in tqdm.tqdm(range(1000), desc="Transfer progress", ncols=100, bar_format='{l_bar}{bar}|'):
        sleep(.1)
        if msvcrt.kbhit():
            if msvcrt.getwche() == ' ':
                interrupt()
                break

def interrupt():
    type("File transfer interrupted, to restart the transfer, type 'restart'")

https://github.com/tqdm/tqdm/issues/375 – sirex Apr 07 '18 at 18:37 — sirex, Apr 07 '18 at 18:37
https://github.com/tqdm/tqdm/issues/313 – Evandro Coan May 05 '19 at 21:53 — Evandro Coan, May 05 '19 at 21:53
Is there a place that explains WHY this happens? – Eduardo Reis Feb 18 '21 at 18:32 — Eduardo Reis, Feb 18 '21 at 18:32

SciPy · Accepted Answer · 2019-10-31T13:17:45.137

165

Try with position=0 and leave=True

(Solution working in Google Colab to avoid printing to a newline)

from tqdm import tqdm 
import time

def foo_():
    time.sleep(0.3)
range_ = range(0, 10)
total = len(range_)

with tqdm(total=total, position=0, leave=True) as pbar:
   for i in tqdm((foo_, range_ ), position=0, leave=True):
    pbar.update()

edited Oct 31 '19 at 13:17

answered Jul 17 '19 at 09:26

SciPy

5,412
4
18
18

17

If you make a colab cell with `from functools import partial`, `from tqdm import tqdm`, `tqdm = partial(tqdm, position=0, leave=True)` at the top of your notebook, this will set it up so you can just call `tqdm(iterable)` and it will work nicely in colab. – Robert Bracco Jul 27 '20 at 15:13
2

`for i in tqdm(range(n), position=0, leave=True)` also works for me in PyCharm – Blupon Oct 01 '21 at 13:37
This also works in both jupyterlab and terminal. – LI Xuhong Oct 20 '21 at 06:43
2

instead of tqdm.tqdm. Use tqdm.auto.tqdm – Vivasvan Patel Nov 07 '21 at 14:01
1

Will this work if tqdm is used internally by a package? I'd guess no since the package would have it's own `from tqdm import tqdm` statement or similar. – Sterling Jun 20 '22 at 18:58
1

I didn't need to set `leave=True`. This seems to be the default. – winderland Sep 05 '22 at 23:37

Laurent GRENIER · Answer 2 · 2020-01-14T18:21:23.207

35

tqdm_notebook is deprecated. You must use tq.notebook.tqdm instead.

import tqdm.notebook as tq
for i in tq.tqdm(...):

Furthermore, tqdm_notebook was really miserable in terms of performances. That's fully corrected with the new library.

edited Jan 14 '20 at 18:21

answered Jan 14 '20 at 18:10

Laurent GRENIER

612
1
6
13

This displays no progress bar in a jupyter Lab session – 00__00__00 Oct 15 '20 at 12:07
How did you works with `asyncio` without `tqdm.asyncio`? – huang Jul 12 '21 at 07:36
@00__00__00 you´d need to install / update ipywidgeds and activate it [Widgets](https://ipywidgets.readthedocs.io/en/stable/user_install.html) ```bash pip install ipywidgets jupyter nbextension enable --py widgetsnbextension ```s – MisterMushn Jun 14 '22 at 16:55

José Vicente · Answer 3 · 2020-06-23T10:59:53.107

35

I have realized that closing tqdm instances before using tqdm again fixes the problem of printing status bar in a new line on Jupyter Lab:

while len(tqdm._instances) > 0:
    tqdm._instances.pop().close()

Or even better, thanks to Nirmal for the suggestion:

tqdm._instances.clear()

edited Jun 23 '20 at 10:59

answered Apr 15 '20 at 13:43

José Vicente

451
4
4

6

Damn, this whole time I've been restarting Jupyter notebooks to fix the multiline stuff. This is really helpful, thanks! – Jay Mody May 26 '20 at 04:04
this solved it! – Vaaal88 Feb 02 '22 at 13:41
This is the best solution. Thanks!! – DrRDN Jun 08 '23 at 16:47

RAHUL TIWARI · Answer 4 · 2021-09-22T01:23:13.317

34

I faced this problem a lot and sometimes position = 0 & leave = True does not works. So, I found one alternate way.

Instead of tqdm.tqdm you can use tqdm.auto.tqdm
or
instead of

from tqdm import tqdm

try using

from tqdm.auto import tqdm

edited Sep 22 '21 at 01:23

answered May 04 '21 at 08:44

RAHUL TIWARI

560
4
5

3

This formats nicely in Jupyter Notebooks/Jupyter Lab as well – Daniel Bourke Mar 29 '22 at 01:47
This works in VS Code Notebooks too. – sea the mountains Aug 03 '23 at 23:17

score 19 · Answer 5 · answered Dec 15 '19 at 16:00

19

from tqdm import tqdm_notebook

this command works in google colab.

answered Dec 15 '19 at 16:00

Aptha Gowda

968
2
10
20

score 7 · Answer 6 · 2019-01-14T10:06:02.463

Try using tqdm.tqdm.write in place of the standard print()

This will print above the progress bar and move the progress bar one row below.

I tested this using below code, pressing space will print into stdout but not break the loop. It was not 100% clear what you are trying to achieve, since the interrupt() function of yours only checks the type of the provided string. type() built-in function

import tqdm
import msvcrt
from time import sleep

def transfer():
    for i in tqdm.tqdm(range(1000), desc="Transfer progress", ncols=100, bar_format='{l_bar}{bar}|'):
        sleep(.1)
        if msvcrt.kbhit():
            if msvcrt.getwche() == ' ':
                interrupt()
                # break

def interrupt():
    tqdm.tqdm.write("File transfer interrupted, to restart the transfer, type 'restart'", end="")

transfer()

EDIT: to include end parameter of tqdm.write() as noted by Paul Netherwood tqdm.tqdm.write()

Note that tqdm.write() has an optional parameter 'end' which defaults to '\n' which will cause a line wrap. Set end='' — Paul Netherwood, Jan 11 '19 at 16:25

score 7 · Answer 7 · edited Jun 11 '21 at 09:21

7

from tqdm import notebook

Instead of tqdm(looping)
Use notebook.tqdm(looping)

edited Jun 11 '21 at 09:21

Hamman Samuel

2,350
4
30
41

answered Jun 02 '20 at 10:32

code-freeze

465
8
8

score 4 · Answer 8 · answered Apr 29 '20 at 07:05

4

You might have imported tqdm twice. Restart the whole notebook kernel and run again. It will solve the issue. It might also be showing because of any print statements inside the tqdm

answered Apr 29 '20 at 07:05

Vijeth Rai

321
2
10

score 4 · Answer 9 · answered May 21 '22 at 08:54

4

If you encounter this problem while using tqdm on colab or jupyter notebook, then ...

Use notebook/colab version of tqdm

>>> from tqdm.notebook import trange, tqdm
>>> for i in trange(1000):
...     ...

answered May 21 '22 at 08:54

Shaida Muhammad

1,428
14
25

score 3 · Answer 10 · answered Aug 16 '19 at 23:53

The following is hacky, but seems to work reasonably well to reset tqdm:

from tqdm import tqdm as tqdm_base
def tqdm(*args, **kwargs):
    if hasattr(tqdm_base, '_instances'):
        for instance in list(tqdm_base._instances):
            tqdm_base._decr_instances(instance)
    return tqdm_base(*args, **kwargs)

Sometimes previous output is printed at the start (which I am not sure how to remove), but I find it much less annoying than newlines (especially in long loops).

This works very well, thank you – Cerberton Sep 29 '21 at 18:36 — Cerberton, Sep 29 '21 at 18:36

score 3 · Answer 11 · answered Nov 17 '19 at 15:23

Import tqdm.

from tqdm import tqdm

First start the code, where you use tqdm, stop it because of multiple lines output.

Then do:

list(getattr(tqdm, '_instances'))

for instance in list(tqdm._instances):
    tqdm._decr_instances(instance)

If you get an error:

AttributeError: type object 'tqdm' has no attribute '_instances'

You need first to start your code, where you use tqdm and only after that start code which mention.

And after all this manipulations your tqdm will work fine.

score 3 · Answer 12 · answered Jan 19 '21 at 09:32

Besides the aforementioned position=0, leave=True parameters, in my case the tqdm's default ascii=False parameter was also printing on new lines after a few iterations. You easily identify if this is the case by looking at the progress bar: if there are any weirdly formatted symbols (e.g. question marks) in your progress bar, you should try using ascii=True.

So this worked for me:

from tqdm.auto import tqdm
...

with tqdm(data, position=0, leave=True, ascii=True) as iterator:
   for x in iterator:
      # do stuff
      ...

      iterator.set_postfix_str(msg)

score 2 · Answer 13 · answered Aug 26 '20 at 20:14

2

leave=False for the inner loop worked in my case.

for j in tqdm(outer_list):
    for i in tqdm(inner_list, leave=False):

Evnironment with tqdm==4.38.0 and Python 3.6.7

answered Aug 26 '20 at 20:14

Danil

4,781
1
35
50

Works also for multiple loops in the terminal, not in notebook (they all stay in place) – KSHMR Jan 07 '23 at 10:23

score 2 · Answer 14 · answered Apr 17 '22 at 20:50

2

The issue I'm having might not be common, but in case it's useful to anyone, I was printing another variable to the console. Disabling that print fixes the issue.

answered Apr 17 '22 at 20:50

Yupeng Tang

663
6
7

1

THIS IS KEY! If you have any print statement, it will mess things up. – user3180 Jun 01 '22 at 04:48

score 0 · Answer 15 · answered Mar 21 '19 at 15:41

0

Try using tqdm.tnrange()

for i in tqdm.tnrange(len(df)):

answered Mar 21 '19 at 15:41

ASHu2

2,027
15
29

3

Please include code as codeblocks instead of images so as to make it easier to copy/paste (and also reduce Stack Overflow's hosting cost) – Nino Filiu Mar 21 '19 at 16:01
1

Ah sorry, but the code just `for i in tqdm.tnrange(len(df)):` rest is for jupyter as printing in newline also happens inside notebook. – ASHu2 Mar 21 '19 at 16:50

Yury Wallet · Answer 16 · 2019-08-26T21:13:02.377

I've tried tqdm solution but as I'm using Spyder (Anaconda) it doesn't work in my case as supposed due to mentioned in other answers conflict between write and print commands. I came up with simple and working although not fanciest solution.

def ybar(progr, total, step=50):
    #starts with 1
    l2=(progr/total)//(1/step)
    if progr==1: print(f'[{total}]: '+'|'*int(l2), end = '') 
    else:
        l1=((progr-1)/total)//(1/step) 

        ll=int(l2-l1)
        if l1 < l2: 

            for j in range(1,ll+1):
                if (int(l1)+j)%5==0:
                    print('*', end = '')
                else:
                    print('|', end = '')
        if progr==total: print("  DONE")

And as result you'll get simple: [100]: ||||||

for i in range(1,101):
    ybar(i,len(range(1,101)),50)
    #something

There are plenty of solutions here: Python Progress Bar

score 0 · Answer 17 · answered Nov 03 '20 at 02:34

0

https://github.com/tqdm/tqdm#parameters I think sometimes tqdm cannot catch the screen width use ncols=xx to restrict line width ex: tqdm(iter,ncols=80): # restrict line width=80

answered Nov 03 '20 at 02:34

x8569

1

score 0 · Answer 18 · answered Jun 02 '23 at 09:09

Although the original question looks to be for a Linux shell. From the Windows command line, the following worked.

from tqdm import tqdm

for x in tqdm([1,2,3,4], leave=True, ncols=80, position=0):
    pass
    for y in tqdm([1,2,3,4], leave=False, ncols=80, position=1):
        pass

I feel that if i'd read the documentation, rather than blindly trying all the suggested fixes, i'd have sorted this much quicker!

score -8 · Answer 19 · answered Oct 15 '17 at 16:31

-8

Try from tqdm import tqdm_notebook as tqdm instead of from tqdm import tqdm.

answered Oct 15 '17 at 16:31

Imran

608
10
17

1

really? That's for jupyter/ipython notebooks and really doesn't seem applicable here. – Winston Ewert Oct 15 '17 at 16:37

Why is tqdm printing to a newline instead of updating the same line?

19 Answers19

Linked

Related