Does using print() too much cause it to fail?

Question

TL;DR:

The print() result is not updating in a Windows Console. Executes fine in IDLE. Program is executing even though Windows Console is not updating.

Background

I have a file, test.py that contains:

Edit: Included the conditions that I used to see if the Console was updating. Eventually the series of X values never prints again in Console and the Console never scrolls back up (as it normally does when output is being generated at the bottom).

count = 0
while True:
    print ("True")
    count += 1
    if count == 10:
            print ("XXXXXXXXX")
            count = 0

When I run this in cmd.exe it obviously prints a very large number of True.

However, after about 25 seconds of running, it stops printing any more, though the program is still running and can be seen in the Task Manager.

I have a program with some progress indicators that end up stay at say 50% even though they are moving well beyond 50% simply because print() is not showing in the Console output.

Edit: The true use case problem.

The above code was just a test file to see if printing in Console stopped in all programs, not the one I was running. In practice, my program prints to Console and looks like:

line [10] >> Progress 05%

Where line [10] isn't real but I merely typed here to show you that print() sends to that line in the Console window. As my program continues it increments:

line [10] >> Progress 06%
line [10] >> Progress 11%
.
.
.
line [10] >> Progress 50%

Each time line [10] is overwritten. I use ANSI escape characters and colorama to move the Console cursor accordingly:

print('\x1b[1000D\x1b[1A')

This moves the cursor 1000 columns left and 1 row up (so the start of the previous line).

Something is happening where the print("Progress " + prog + "%") is not showing up anymore in Console because eventually the next bit of Python gets executed:

line [11] >> Program Complete...

I verified the resultants which get put into a folder. So the program continued to run while the Console did not update.

Edit: Here is the script running the updates to the stdout.

def check_queue(q, dates, dct):
    out = 0
    height = 0

    # print the initial columns and rows of output
    # each cell has a unique id 
    # so they are stored in a dictionary
    # then I convert to list to print by subscripting
    for x in range(0, len(list(dct.values())), 3):
        print("\t\t".join(list(dct.values())[x:x+3]))
        height +=1 # to determine where the top is for cursor

    while True:
        if out != (len(dates) * 2):
            try:
                status = q.get_nowait()
                dct[status[1]] = status[2]
                print('\x1b[1000D\x1b[' + str(height + 1) + 'A')

                # since there was a message that means a value was updated
                for x in range(0, len(list(dct.values())), 3):
                    print("\t\t".join(list(dct.values())[x:x+3]))

                if status[0] == 'S' or 'C' or 'F':
                    out += 1
            except queue.Empty:
                pass
    else:
        break

In short, I pass a message to the queue from a thread. I then update a dictionary that holds unique cell IDs. I update the value, move the cursor in Console to the upper left position of the printed list, and print over it.

Question:

When using stdout, is there a limit to how many times you can print to it in a period of time?

Python certainly has no limits on how much output a process can generate. — President James K. Polk, Aug 08 '17 at 17:22
@user2357112 Take a look at the code block now. I updated it with how I tested to make sure of this. What ends up happening is eventually the series of `X` never appears again and the console never scrolls the previous output up. This leads me to believe its not longer printing. — , Aug 08 '17 at 17:22
@user2357112 So what I may need to ask instead is "Why is my Windows Console not updating with the print() result?". Thoughts? — , Aug 08 '17 at 17:23
See [Is there a limit on the output of Console Window](https://stackoverflow.com/questions/34160977/is-there-a-limit-on-the-output-of-console-window) — President James K. Polk, Aug 08 '17 at 17:24
@JamesKPolk My screen buffer size is already at 9999. The above code is just a test, but my program writes to the same line using ANSI escape sequences. So I never get even close to line 9999. — , Aug 08 '17 at 17:27
Well, this problem is not encountered in ubuntu!!! It keeps on printing untill CTRL+Z is not pressed which lead to the affirmation that python has no limits on how much output a process can generate@James K Polk — , Aug 08 '17 at 17:30
I like to give a suggestion that please run the code on any **IDE** and then check for the same — , Aug 08 '17 at 17:36
@Mandy8055 We think alike. I ran this in IDLE with **NO** problem at all. Hence my question is directed at the Windows Console environment. — , Aug 08 '17 at 17:38
I am damn sure that this is the limitation imposed by the OS — , Aug 08 '17 at 17:39
@Mandy8055 Any familiarity with the Windows OS and altering the `stdout` limits? — , Aug 08 '17 at 17:41
@Mandy8055 Unfortunately no. What that is saying that the longest string you can have after issuing a `.exe` is 8191 characters. So all your passed arguments to a `.exe` cannot be more than that. — , Aug 08 '17 at 17:47
I tested the above code in windows 10 with all invariant. It is working fine — , Aug 08 '17 at 18:12
When you say you use `print('\x1b[1000D\x1b[1A')` do you use it within the script? If so could you include it in your simplified script as you would use it in the "more complicated one"? — MSeifert, Aug 08 '17 at 18:19
Could you provide some (even if they are just random) inputs for this function as well? Would make it easier to run it by simply copy&pasting it. — MSeifert, Aug 08 '17 at 18:31
@MSeifert All you need is a dictionary where the key == value. So each key is identical to its value. What I do is change the color associated with it by using `colorama`. Unfortunately Python resides on a different system then the one I use to get on SO. — , Aug 08 '17 at 18:34
and what about `q` and `dates`? What should these be? Please have a look at [mcve]. Currently it's one snippet that works as expected and one snippet that cannot be run without knowing the inputs (or at least "rough inputs"). Is there a way you can put the `print('\x1b[1000D\x1b[' + str(height + 1) + 'A')` in the first snippet so it shows the problem? — MSeifert, Aug 08 '17 at 18:36
Support for ANSI escape sequences in Windows is brand new. It isn't entirely implausible that there might be a bug there. But we really, really need a [mcve]. (Emphasis on *minimal*, the *smallest* program that reproduces the problem.) — Harry Johnston, Aug 09 '17 at 02:17
@HarryJohnston, the OP is using colorama, which implements virtual terminal support in Windows via the console API. It's still the preferred way to get VT support in Windows Python, and will be for a long time to come. — Eryk Sun, Aug 09 '17 at 08:32

MSeifert · Accepted Answer · 2017-08-08T17:26:35.263

3

That may well be an illusion (maybe because there's a maximum limit of lines in the console and new ones just replace the first ones then).

There's definetly no limit how much you can print. You could verify this with something that changes each iteration, for example a loop that counts the number of iterations:

import itertools
for i in itertools.count():
    print(i, "True")

edited Aug 08 '17 at 17:26

answered Aug 08 '17 at 17:25

MSeifert

145,886
38
333
352

I am starting to think that it may be the case of reaching number of lines. However, I am overwriting lines in the same spot. Is it double counting? – Aug 08 '17 at 17:26
I don't understand what you mean. Some consoles work like FIFO queues, when a certain limit is reached they discard the oldest line to print the newest one. Like I mentioned in the answer. But in any case could you update your question if this doesn't solve your question and explain **why** it doesn't solve your issue? – MSeifert Aug 08 '17 at 17:30

score 2 · Answer 2 · answered Aug 09 '17 at 00:53

I cannot reproduce the problem in Windows 10 using 64-bit Python 3.6.2 and colorama 0.3.9. Here's the simple example that I tested:

import colorama
colorama.init()

def test(M=10, N=1000):
    for x in range(M):
        print('spam')
    for n in range(N):
        print('\x1b[1000D\x1b[' + str(M + 1) + 'A')
        for m in range(M):
            print('spam', m, n)

Each pass successfully overwrites the previous lines. Here's the final output from looping N (1000) times:

>>> test()
spam 0 999
spam 1 999
spam 2 999
spam 3 999
spam 4 999
spam 5 999
spam 6 999
spam 7 999
spam 8 999
spam 9 999

If this example fails for you, please update your question and include the versions of Windows, Python, and colorama that you're testing.

I am using Windows 7, Python 3.6.0 and the latest version of colorama. What is interesting is even without using an coloroma capability, eventually the stdout stops updating in the Console (for example my `while` loop in the question). — , Aug 09 '17 at 12:45

score 1 · Answer 3 · answered Aug 08 '17 at 17:27

Sounds like it might be a system limitation, not a Python process issue? I've never run across a 'hang' related to print statements (or any built-in function), however you may want to look at mapping performance and memory usage:

High Memory Usage Using Python Multiprocessing

As far as how many times you can print in a period of time, that is almost exclusively based on the speed the system executes the code. You could run some benchmark tests (execution time / number of executions) across several platforms to test performance with specific system specs, but I'd say the likely cause of your issue is system / environment related.

Does using print() too much cause it to fail?

3 Answers3