6

I can't get the following script to work without throwing an EOFError exception:

#!/usr/bin/env python3

import json
import sys

# usage:
# echo '[{"testname": "testval"}]' | python3 test.py

myjson = json.load(sys.stdin)
print(json.dumps(myjson))

answer = input("> ")  # BUG: EOFError: EOF when reading a line
print(answer)

I've read this question which seems to be related: Python STDIN User Input Issue

I think that tells me I need to clear the stdin buffer ? But I'm not sure how because print(sys.stdin.readline()) just outputs a newline and the EOFError is still there.

I also tried using the sys.stdin.flush() method (found in this question: Usage of sys.stdout.flush() method) although I still don't understand what it does because I couldn't find it in the official documentation (3.6), the closest I found was this but it doesn't mention flush: https://docs.python.org/3/library/sys.html

Please bear in mind that I'm not a programmer nor do I have a CS education or background. I just write scripts to automate parts of my, otherwise non-technical, work. So if you know any good beginner ressource on how stdin/stdout works in the shell with Python please do tell.

rmercier
  • 155
  • 1
  • 10
  • `json.load(sys.stdin)` consumed all the data available on stdin. There's nothing left to read, hence EOF. – melpomene Sep 09 '17 at 11:00

2 Answers2

5

By piping input, Python is opening sys.stdin as a FIFO. Otherwise, Python will open sys.stdin to /dev/tty or equivalent.

You can verify this with:

import os,sys,stat
print("isatty():", sys.stdin.isatty())
print("isfifo():", stat.S_ISFIFO(os.fstat(0).st_mode))

Run this twice, once piping in data, once not.

I get:

$ echo "Test" | ./test2.py
isatty(): False
isfifo(): True

$ ./test2.py
isatty(): True
isfifo(): False

So your EOF occurs because the FIFO sys.stdin is opened to is empty.

You can reopen sys.stdin to /dev/tty, however.

j = json.load(sys.stdin)
print(j)

sys.stdin = open("/dev/tty")

answer = input("> ")
print(answer)

Which would work fine:

$ echo '{"key":"val"}' | python3 ./testjson.py
{'key': 'val'}
> testing
testing
jedwards
  • 29,432
  • 3
  • 65
  • 92
  • Haha I should've put the "not a programmer" part in bold, there's a lot of things I'll have to research to understand what is happening but it works perfectly. Thanks a lot for the explanation and solution! – rmercier Sep 09 '17 at 11:30
  • Please don't worry about not getting this (really). This sort of thing *rarely* comes up and when it does is incredibly confusing. – jedwards Sep 09 '17 at 11:32
  • thank you so much @jedwards, I wish SO and Google showed your answer first, before I spent hours to find your excellent solution :) – Edgar Manukyan Dec 18 '19 at 03:52
0

You can't really "clear" the standard input (stdin). It is there, available for reading and many input functions read until end of file (EOF, or the end of standard input). The "flush" operation is for standard output.

In your case, the json.load(sys.stdin) operation will read the entire standard input (and then it will be closed). At that point, no more input is available.

If you both want to read input data AND have interactive input from the user, consider reading your data from a file and using standard input only for interactive user input.

payne
  • 13,833
  • 5
  • 42
  • 49
  • Thank you for this thourough explanation! I'm writing a script that needs to be executed after a third-party program that exports a JSON string to stdout. And I can't change how this third-party program operates (by making it write the JSON to a file for example). But I also need the user to manually accept some JSON dicts and discard others. Is there no way to do something like that in Python? – rmercier Sep 09 '17 at 11:15
  • Perhaps you pipe the JSON output from the third-party script to a temporary file, and then give that file to your program. You generally can't use standard input to BOTH read a file and read user input, in Python or any language. – payne Sep 09 '17 at 11:28