9

I want to be able to debug a Python (Django) application with pdb under uWSGI, and I'm basically having the same issue as described here getting:

  ...
  File "/usr/lib/python2.7/bdb.py", line 49, in trace_dispatch
    return self.dispatch_line(frame)
  File "/usr/lib/python2.7/bdb.py", line 68, in dispatch_line
    if self.quitting: raise BdbQuit
BdbQuit

The difference is that I have a different uWSGI setup, and seems like I cannot make uWSGI to honour-stdin as suggested in the accepted answer from the question above.

My setup is as follows:

1) I have a systemd process to start uWSGI in Emperor mode

[Unit]
Description=uWSGI Emperor service

[Service]
ExecStart=/usr/local/bin/uwsgi --ini /etc/uwsgi/emperor.ini
Restart=always
KillSignal=SIGQUIT
Type=notify
NotifyAccess=all

[Install]
WantedBy=multi-user.target

2) The /etc/uwsgi/emperor.ini looks like this:

[uwsgi]
emperor = /etc/uwsgi/sites
uid = www-data
gid = www-data
limit-as = 1024
logto = /tmp/uwsgi-emperor.log
# I've tried adding both honour-stdin 
# and daemons-honour-stdin here
honour-stdin = true
daemons-honour-stdin = true

3) A sample configuration of one of the uwsgi sites look like this:

#/etc/uwsgi/sites/testproject.ini
[uwsgi]
module = wsgi
chdir = /home/myuser/projects/testproject
home = /home/myuser/.virtualenvs/testproject
env = DJANGO_SETTINGS_MODULE=testproject.settings.dev
daemonize = /tmp/uwsgi-testproject.log
master = true
processes = 1

socket = /tmp/testproject-dev.sock
chmod-socket = 664
vacuum = true

# I've also tried adding both honour-stdin 
# and daemons-honour-stdin here
honour-stdin = true
daemons-honour-stdin = true

4) I'm not sure if its related to the issue, but I also have an nginx configuration to serve the site, it look like this:

upstream app-testproject-dev {
    server unix:///tmp/testproject-dev.sock;
}

server {

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    listen 80;
    server_name dev.testproject.com;

    location / {
        uwsgi_pass app-testproject-dev;
        include /etc/nginx/uwsgi_params;
    }
}

A temporary solution for now is to use remote-pdb as an alternative to my approach, but I'm interested to understand whats the issue in my current configuration setup and how to fix it.

UPDATE: I just realized that even if this work, maybe I'm not properly opening my log files, so that pdb can wait for my input. Right now I'm using tail to see whats going on with the logs, but no idea if this works with pdb?

UPDATE2: Did some more testing, tried to skip the systemd + the uwsgi emperor mode from of the equation by starting the daemon myself via:

sudo /usr/local/bin/uwsgi --ini /etc/uwsgi/sites/testproject.ini

what I'm noticing is that w/o daemonize = /tmp/uwsgi-testproject.log in the .ini file everything works fine, but as soon I daemonize it, stdin starts pointing to /dev/null (I have both honour-stdin and daemons-honour-stdin set to true). I'm checking this with 

ls -l /proc/<proc_id>/fd/0
Todor
  • 15,307
  • 5
  • 55
  • 62
  • Try this: (https://code.google.com/archive/p/modwsgi/wikis/DebuggingTechniques.wiki) – Kousic Dec 10 '18 at 05:54
  • Thank you @Kousic unfortunately modwsgi is an Apache plugin, while uWSGI is a web server, however I will give a try to the custom `Debugger` class which they mention there, but I'm not very optimistic. – Todor Dec 10 '18 at 06:29
  • Did you find anything positive? – Kousic Dec 10 '18 at 09:31
  • pdb needs a terminal, a daemon process does not have one, what terminal are you expecting to interact with? – georgexsh Dec 11 '18 at 04:49
  • @georgexsh as I said I use `tail` in order to attach myself to the program output(`stdout`), at first I didn't realized that I can't use it for `stdin` too. Did some googling and tried a few things, but nothing have worked. Something that make me think that `--honour-stdin` is still not respected by uwsgi is that when I do `ls -l /proc//fd/0` still points to `/dev/null`. – Todor Dec 11 '18 at 06:10
  • @Todor Try using FIFO instead of STDIN to provide input. – Kousic Dec 11 '18 at 06:28
  • @Todor I hope this could help: [uWSGI works as process but not as daemon](https://stackoverflow.com/questions/17099114/uwsgi-works-as-process-but-not-as-daemon) – Kousic Dec 11 '18 at 09:37

1 Answers1

3

I don't know this could help you solve your problem. But, give it a try to configure your uWSGI with the below code.

Into /etc/systemd/system/emperor.uwsgi.service put

[Unit]
Description=uWSGI Emperor
After=syslog.target

[Service]
ExecStart=/usr/bin/uwsgi --ini /etc/uwsgi/emperor.ini
Restart=always
KillSignal=SIGQUIT
Type=notify
StandardError=syslog
NotifyAccess=all

[Install]
WantedBy=multi-user.target

Into /etc/uwsgi/emperor.ini put:

[uwsgi]
master = true
procname-master = Emperor

; Look for vassal configs using this pattern
emperor = /srv/apps/*/uwsgi.ini
; Don't resolve symlinks when considering reload-on-touch
emperor-nofollow = true

; Lowest privs
uid = www-data
gid = www-data

; Clean up our workers when we die.
no-orphans = true

A config for each of the sites (called "vassals" in uWSGI terms).

Into /etc/uwsgi/vassal.ini put:

[uwsgi]
master = true
procname-master = %c

; Run with lower privs
uid = www-data
gid = www-data

; :0 lets the OS assign a port
socket = 127.0.0.1:0
; Register with the FastRouter the list of hostnames
subscribe-to = 127.0.0.1:9001:@hostnames.txt

; Paths are referenced relative to where the INI file is found
chdir = %d

# Task management
; Max 4 processes
processes = 4
; Each running 4 threads
threads = 4
; Reduce to 1 process when quiet
cheaper = 1
; Save some memory per thread
thread-stack-size = 512

# Logging
plugin = logfile
req-logger = file:logs/request.log
logger = file:logs/error.log
log-x-forwarded-for = true

# Python app
plugin = python
virtualenv = venv/
pythonpath = code/
module = %c.wsgi
enable-threads = true

# Don't load the app in the Master.
app-lazy = true

Finally, the Fastrouter. This is like a smart load balancer, but will distribute requests according to their hostnames.

Into /etc/uwsgi/router.ini put

[uwsgi]
master = true
procname-master = FastRouter

uid = www-data
gid = www-data

plugin = fastrouter
fastrouter = %d/server.sock
; run as lower privs
fastrouter-uid = www-data
fastrouter-gid = www-data
; handle the scale
fastrouter-processes = 2
; but scale down when quiet
fastrouter-cheap = true
; let others vassals subscribe to us
fastrouter-subscription-server = 127.0.0.1:9001

# Logging
plugin = logfile
req-logger = file:logs/request.log
logger = file:logs/error.log

With the subscription server, The vassals can register with the FastRouter, telling it which hostnames they can handle, and what port they can be contacted on.

Then start it up using:

# systemctl start emperor.uwsgi

If this works without issue, enable it to start on boot:

# systemctl enable emperor.uwsgi
Kousic
  • 2,651
  • 2
  • 10
  • 23