How to sequentially process multiple stdin inputs to python?

Question

I have a process that prints to stdout at regular intervals (in seconds granularity), and I'd like to pipe the output of that process to a python program via stdin to process it. The problem I face is that I'd also like to take input from the user on how to process it before moving on. As a toy example,

The program (interval.sh) that prints to stdout in regular intervals.:

#!/bin/bash

for i in {1..10}
do
    echo $i
    sleep 1s
done

Python program (test.py) to process the input:

#!/usr/bin/env python3

import sys

for line in sys.stdin:
    while True:
        validate = input("Do you want to accept task {}? [y/n]\n".format(line))
        if validate == 'y':
            print("User accepted the input\n")
            break
        elif validate == 'n':
            print("User rejected the input\n")
            break
        else:
            print("Please enter a valid input")

I currently run the program like:

$ ./interval.sh | ./test.py
Do you want to accept task 1
? [y/n]
Input not supported
Do you want to accept task 1
? [y/n]
Input not supported
Do you want to accept task 1
? [y/n]

As you can see, the above program thinks that the input from the shellcode is the user's input. What I would like to do is:

$./interval.sh | ./test.py
Do you want to accept task 1? [y/n] y
The User accepted the input
Do you want to accept task 2? [y/n] y // Move to task 2 only when the user provides a valid input
The User accepted the input
Do you want to accept task 3? [y/n]

I see the problem because the input from the program and user are from stdin and hence difficult to differentiate. Furthermore, interval.sh cannot be modified in my actual scenario. How else can I approach this problem?

I would recommend running your shell script from within python (subprocess) and reading it's output in your code. This way any input you send will be handled by python instead of script first (It is your choice to send it to script or not) https://stackoverflow.com/questions/4760215/running-shell-command-and-capturing-the-output — glory9211, Jan 19 '21 at 17:42
It seems like subprocess would wait until the script completes its full execution. Would I be able to read the output from the script as it prints out and process it one after another? — 0x0, Jan 19 '21 at 17:56
Does the subprocess blocks your code using above method ? I believe there should be a way to make it non-blocking. Will have to look it up — glory9211, Jan 19 '21 at 17:59
Yeah. It prints 1 through 10 first and then moves on to asking the user for input. — 0x0, Jan 19 '21 at 18:01
Have a look at https://stackoverflow.com/questions/16071866/non-blocking-subprocess-call — glory9211, Jan 20 '21 at 01:45

score 0 · Answer 1 · answered Jan 19 '21 at 09:36

This very likely has something to do with the way you're using pipes. command_1 | command_2. The output from command_1 acts as an input for command_2 and this is why your program fails.

A very good alternative would be to write the entirety In python.

Another thing you can try is

for i in {1..10}
do
    echo $i
    sleep 1s
    ./test.py $i
done

With this change to your bash script, you no longer need to use the pipe method. Simply running the bash script would cause the python program to run every time and thus the bash code is executed and it would also fix the issue of input.

import sys

line = sys.stdIn    

while True:
    validate = input("Do you want to accept task {}? [y/n]\n".format(line))
    if validate == 'y':
        print("User accepted the input\n")
        break
    elif validate == 'n':
        print("User rejected the input\n")
        break
    else:
        print("Please enter a valid input")

Now you can just run the bash script

./interval.sh

Thanks! interval.sh was only a toy example. In my scenario I don't really have control over that program. I will edit my question to add that info. — 0x0, Jan 19 '21 at 17:02

How to sequentially process multiple stdin inputs to python?

1 Answers1