Keras forces output to sys.stderr (for which a fix was rejected on GitHub). There seems to be a problem writing to system outputs from a child process in a Web App. This leads to my code throwing the appropriate error when attempting to inform about the backend implementation when Keras is imported.
AttributeError: 'NoneType' object has no attribute 'write'
I tried to redirect output to os.devnull according to this answer before instantiating a Flask application and starting it with a web.config. However, the error persisted. Curiously, writing output without multiprocessing worked just fine.
import sys
from flask import Flask
import keras
app = Flask(__name__)
@app.route('/')
def main():
print('Hello!')
sys.stdout.write('test\n')
sys.stderr.write('emsg\n')
return 'OK.', 200
Even from keras import backend as k works. That's the statement that originally produced the error. This left me baffled. What could possibly be the matter?
Minimal example
In my application, a sub process is spawned for training models. When trying to write output within the multiprocessing.Process, an error is thrown. Here's some code to reproduce the situation.
import sys
from flask import Flask
from multiprocessing import Process
def write_output():
sys.stdout.write('hello\n')
def create_app():
apl = Flask(__name__)
Process(target=write_output).start()
@apl.route('/')
def main():
return 'OK.', 200
return apl
This application is then instantiated in another file and called from web.config. Basic logging confirmed the error was still being thrown.
Almost a fix
Although not a fix, I got the system working using threading. By simply switching multiprocessing.Queue and Process to queue.Queue and threading.Thread, no errors like above are thrown. For my use case this is acceptable for now. Of course it's not a solution to the problem of writing output in a child process.
