How to exhaustively see queries Django is running?

Question

I am optimizing a big part of my code so I am trying to reduce the database hits to the minimum and try to grab all necessary data in big but few queries and cache it.

However I have not found an effective way of seeing whether Django is actually running a query against the database or whether it is returning cached or prefetched data.

Ideally, I want to debug the relevant parts of my code line by line and each raw SQL query should be displayed as they are executed. Essentially some kind of listener that logs every query that is run as soon as it is run.

This way to check running queries suggested by the docs is not good enough, as it only displays queries formed in the same file where you access connection.queries, apparently.

Accessing QuerySet.query is no good either, as it requires a, well, QuerySet object. There are other scenarios where queries are run against the database but the return type is not a QuerySet.

I am not sure the Django Debug Toolbar would work either, because I don't want my behavior wrapped in a view, I want to execute an independent script and thoroughly debug it.

did you try django log https://stackoverflow.com/a/20161527/8060120 — Brown Bear, Jun 06 '18 at 12:50
@BearBrown that seems to fit what am I asking. Is there any way of enabling or disabling those logging options in the code so I can narrow it down to the parts I want to debug? Also, I tried setting up a formatter to color it differently but it is always red, no matter if I target the DEBUG level, INFO, WARNING, ERROR or CRITICAL. — dabadaba, Jun 06 '18 at 15:55
unfortunately i never use it just read your question and the answer — Brown Bear, Jun 06 '18 at 16:00

score 1 · Accepted Answer · answered Jun 06 '18 at 12:59

1

You can try changing LOGGING config, for example in your settings.py (logs all your queries if you DEBUG is True):

LOGGING = {
    'version': 1,
    'filters': {
        'require_debug_true': {
            '()': 'django.utils.log.RequireDebugTrue',
        }
    },
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'filters': ['require_debug_true'],
            'class': 'logging.StreamHandler',
        }
    },
    'loggers': {
        'django.db.backends': {
            'level': 'DEBUG',
            'handlers': ['console'],
        }
    }
}

answered Jun 06 '18 at 12:59

Andrey Bondar

120
7

Is there any way to enable/disable that config right in the middle of the code? And how can I change the color of the logged queries? I tried using a formatter but it doesn't seem to help. – dabadaba Jun 06 '18 at 15:56
Couldn't you enable logging in the db itself? – JL Peyret Jun 06 '18 at 16:03
@JLPeyret honestly I never even thought about that at first, then realized that was probably a possibility. I have never done that though. I'd be still in need of enabling and disabling it though. – dabadaba Jun 06 '18 at 19:12

score 1 · Answer 2 · answered Jun 06 '18 at 14:33

Yes, You should use the django-debug-toolbar for that. It will display each query made on some view, if that code you want is like, inside some Model or other python file and sometime is called for some view. Even if it is as a signal on model, for instance. It will be caught by the debug-toolbar.

Another option is tu use the python manage.py shell to call those queries. You can use the django-extensions with the --print-sql parameter in order to display it.

Or, print the sql it self, I usally do this:

>>> qs = User.objects.filter(username='luan')
>>> print(qs.query)
SELECT "users_user"."id", "users_user"."password", "users_user"."last_login", "users_user"."is_superuser", "users_user"."username", "users_user"."first_name", "users_user"."last_name", "users_user"."email", "users_user"."is_staff", "users_user"."is_active", "users_user"."date_joined", "users_user"."ref_id", "users_user"."created_on", "users_user"."updated_on" FROM "users_user" WHERE "users_user"."username" = luan ORDER BY "users_user"."id" ASC

How to exhaustively see queries Django is running?

2 Answers2