Dynamic multiple lines refresh printing in Python

Question

I am completely new to Python and there's something that I can't figure out how to do.

What I want do to:

I would like to be able to dynamically rewrite the output on multiple lines in the console. I have seen many solutions for a single line output but not that many for multiple lines. The following threads on Stack Overflow are similar, but didn't solve completely my problem (I'll explain later why):

Python - Rewrite multiple lines in the Console

Dynamic refresh printing of multiprocessing or multithreading in Python

Also, I would like to be able to redirect the output to a file or a pipe (let's say that in the solutions provided above, I would run ./script.py > out.txt and if I interrupt that, I want the intermediate progress bar to be in the output file).

Why do I want to do that?

I am trying to parse a continuous Json stream which looks something like that:

{
...valid Json data block...
},{
...Another valid Json data block...
}, etc...

What I would like in the end would be something like (the process for which I need help is the filter to the only one Json block):

./generate-stream | ./filter-single-view | ./json-processing

NB: The fact that I'm using Json doesn't really matter for my problem, I'm mostly saying it to give you a context. The idea is simply to have a piping of scripts that generates multiple data, filters it to have one data at a time and then processes this data.

I have already parsed the stream to have my Json block in a string that I would like to print (and then print the next one that would rewrite on it). Example of a dynamic refresh:

{
"i": 0
}

Console output step 1 (I already have all that information stored in a string)

{
"i": 1
}

Console output step 2

{
"i": 2
}

Console output step 3, etc...

Hence, the answer to the first thread doesn't work with my problem because I can't output the result and the answer to the second doesn't work for the same reason + this way to clear the screens messes up the data, which can't be used properly in the next process. Another thing that would solve my problem would be to find a way to ignore continuously the clear screen character (I've tried ignoring the first characters, it worked for the first input and then was all messed up).

Please tell me if something was unclear. Thanks!

Assuming you want to make something like "top" at the terminal using curses. I would ask there are so many other GUI's that have this problem solved. I would suggest using one of those. Since you are using json I would highly suggest the browser. — Back2Basics, Mar 17 '14 at 21:34
Thank you for your answer. However, I am not quite sure of what you mean... My goal here is not to make a GUI (if I can avoid using one, it'd actually be better), curses came up as a solution to similar problems. — 7hibault, Mar 18 '14 at 09:21

score 0 · Accepted Answer · answered Mar 19 '14 at 12:33

I managed to solve my problem!

I told you the stream format above, don't worry about the parsing details. This code parses the stream and displays the Json data singularly

    import sys
    import json
    import os

    def main(): 
        json_string=""
        try:
            while True:
                line=sys.stdin.readline()
                if line!="},{\n":
                    json_string=json_string+line
                    continue
                else:
                    #Output singular Json data
                    json_string=json_string+"}\n"
                    json_data=json.loads(json_string)
                    json.dump(json_data, sys.stdout) 
                    os.system('cls' if os.name == 'nt' else 'clear')
                    sys.stdout.flush() 
                    json_string="{\n" 
        except KeyboardInterrupt:
            sys.stdin.close()
            print "Interrupted by user. Bye!"  

    if __name__ == "__main__":
        main()

Now, I want this singular data to be read in another script piped behind (stdin). I didn't really understand why, but any readline would be blocking. Thus, I implemented the following

    import sys
    import json
    import os

    k = 0
    parsing=False
    try:
        buff = ''
        while True:
            #The 2 following functions are blocking
            #buff= sys.stdin.readline()
            #buff = json.load(sys.stdin)
            buff += sys.stdin.read(1)
            if buff.endswith('{'):
                parsing=True;
                k+=1
            if buff.endswith('}'):
                k-=1
            if k==0 and parsing:
                print buff
                #The first 7 characters clear the screen
                data=json.loads(buff[7:])
                buff = ''
                parsing=False

    except KeyboardInterrupt:
       sys.stdout.flush()
       pass

Please let me know if you have any comment on that.

Dynamic multiple lines refresh printing in Python

1 Answers1