pyramid subprocess call STDERR and STDOUT trapping logging

Question

I am using a views.py subprocess os call which fails with non-zero return code and I need to trap what is going wrong. I would also like to log everything and have struggled to follow instructions (https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html?awesome > Advanced Configuration > see below) to get this working.

TIA

The pyramid setup was built recently thus

virtualenv --no-site-packages myproj
cd /home/user/myproj/
source bin/activate
pip3.6 install pyramid pyramid-debugtoolbar pyramid-jinja2 waitress cookiecutter
pcreate -s starter mywsgi
cd mywsgi
python3.6 setup.py develop

at run time pserve production.ini

The views.py call

import os
import subprocess
import json
from pyramid.view import view_config

@view_config(route_name='overlay_event', renderer='json')
def event_view(request):
    return {'new_overlay': subprocess.check_output(['/usr/bin/foo', '/path/thing.script',json.dumps(request.json_body)])

production.ini is boilerplate as below. My version below that. I do get console logging for non-zero return code failures for the subprocess.check_output but not what is presumably getting sent to STDOUT and STDERR from the failed script call. My scripts run fine so it must be some environment, path, JSON, ARGV or other problem. But also I just want logging.

When I edit the below per the above logging instructions I get errors about bad handlers. This happens when I progressively roll back my edits to production.ini. Perhaps this is missing some additional include or ? Does view.py need a change too?

Biolerplate

###
# app configuration
# https://docs.pylonsproject.org/projects/pyramid/en/1.9-branch/narr/environment.html
###

[app:main]
use = egg:myproj

pyramid.reload_templates = false
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.default_locale_name = en

###
# wsgi server configuration
###

[server:main]
use = egg:waitress#main
listen = *:6543

###
# logging configuration
# https://docs.pylonsproject.org/projects/pyramid/en/1.9-branch/narr/logging.html
###

[loggers]
keys = root, myproj

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = WARN
handlers = console

[logger_myproj]
level = WARN
handlers =
qualname = myproj

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s:%(lineno)s][%(threadName)s] %(message)s

My edit of production.ini file

[loggers]
keys = root, filelog

[handlers]
keys = console, myproj, filelog

[formatters]
keys = generic

[logger_root]
level = INFO
handlers = console, filelog

[logger_filelog]
class = FileHandler
args = ('%(here)s/myproj.log','a')
level = INFO
formatter = generic

[logger_myproj]
level = WARN
qualname = myproj

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s:%(lineno)s][%(threadName)s] %(message)s

score 0 · Answer 1 · answered Feb 27 '18 at 15:15

0

You should use run instead of check_output.

try:
    result = subprocess.run(
        ['/usr/bin/foo', '/path/thing.script', json.dumps(request.json_body)],
        check=True,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
    )
except subprocess.CalledProcessError as ex:
    output = ex.stdout.decode('utf-8')
    log.error('Call to /usr/bin/foo failed: %s', output)

Also, I would be wary about passing request.json_body directly to the command line, for possible injection attacks.

answered Feb 27 '18 at 15:15

Antoine Leclair

17,540
3
26
18

Thanks for this - in fact it's python 2.7 so it doesn't have .run. I had a shot with a backport of run [link](https://stackoverflow.com/questions/40590192/getting-an-error-attributeerror-module-object-has-no-attribute-run-while) but python did not like the def arguments input=None and check=False saying they are syntax errors. – Craig Feb 28 '18 at 23:55
Ah, ok. Sorry, I can't help you for Python 2, it's been a while I've done anything with it. Good luck! You could probably do it with `popen`, but I can't say for sure if that's possible and if it's the best solution. – Antoine Leclair Mar 01 '18 at 00:20
Okay I see the problem. The suggested backport for 2.7 was in 3+ language, duh. Let me try to get it to work with a bit more hacking. – Craig Mar 01 '18 at 00:39
Okay I just can't get this to work. I always get back CalledProcessError: command= .... return status and no stderr or stdout. It seems I need to override CalledProcessError – Craig Mar 01 '18 at 02:31

pyramid subprocess call STDERR and STDOUT trapping logging

1 Answers1