How do I disable log messages from the Requests library?

Question

By default, the Requests python library writes log messages to the console, along the lines of:

Starting new HTTP connection (1): example.com
http://example.com:80 "GET / HTTP/1.1" 200 606

I'm usually not interested in these messages, and would like to disable them. What would be the best way to silence those messages or decrease Requests' verbosity?

Related: [nginx and gunicorn](https://stackoverflow.com/q/48700404/562769) — Martin Thoma, Feb 09 '18 at 07:41

aknuds1 · Accepted Answer · 2015-12-11T16:15:54.070

704

I found out how to configure requests's logging level, it's done via the standard logging module. I decided to configure it to not log messages unless they are at least warnings:

import logging

logging.getLogger("requests").setLevel(logging.WARNING)

If you wish to apply this setting for the urllib3 library (typically used by requests) too, add the following:

logging.getLogger("urllib3").setLevel(logging.WARNING)

edited Dec 11 '15 at 16:15

answered Jun 14 '12 at 08:59

aknuds1

65,625
67
195
317

5

I have same issue with `pysimplesoap`, and this answer help me save my day – Janith Chinthana Aug 26 '13 at 13:19
2

You could combine the two lines like this: logging.getLogger('requests').setLevel(logging.WARNING) – jpoppe Oct 04 '14 at 21:58
@JasperPoppe Thanks for the suggestion, implemented. – aknuds1 Oct 05 '14 at 15:52
8

I had to add this line for the "urllib3" logger to suppress requests log messages. – dgassaway Jan 29 '15 at 22:42
13

I was need to import logging; logging.getLogger("urllib3").setLevel(logging.WARNING), too. Logger for "requests" doesn't prevent these messages. – m_messiah Feb 12 '15 at 05:28
6

For some reason when using the requests library in python3 you have to do `getLogger("urllib3")` to suppress the messages. – robru Aug 04 '15 at 20:10
@Robru Sounds as if requests is basing itself on urllib3 these days quite simply, which does logging of its own. That's how it goes :) – aknuds1 Aug 04 '15 at 20:12
Is there a way to make change the level of the logging from requests to DEBUG instead of INFO? I would like to keep the messages, but only at the DEBUG level (so they are only in my offline debug log, and not on the screen). – Frank Meulenaar May 10 '16 at 08:53
@FrankMeulenaar I would pose that as an independent question. Anyhow, I guess what I would do is configure the requests stream handler outputting to screen to ignore anything below or equal to INFO. – aknuds1 May 10 '16 at 09:28
I set it to `CRITICAL` but I'm still getting the same log message: `2017-08-05 15:17:13,334 ERROR -- : Certificate did not match expected hostname: www.improving-autonomy.org. Certificate: {'subjectAltName': [('DNS', '*.wordpress.com'), ('DNS', 'wordpress.com')], 'subject': ((('commonName', u'*.wordpress.com'),),)}` Any help will be highly appreciated – Daniel García Baena Aug 05 '17 at 15:29
Solved. Full path was needed: `logging.getLogger("requests.packages.urllib3").setLevel(logging.CRITICAL)` – Daniel García Baena Aug 05 '17 at 16:10
7

Current versions of the `requests` library do not vendor `urllib3` any more and so doesn't log *anything*. You only need to ask the `urllib3` library to not propagate: `logging.getLogger("urllib3").propagate = False`, or increase the logging level if you still need to see warnings or critical messages (only warnings are issued currently). – Martijn Pieters Mar 15 '19 at 13:06

kbrose · Answer 2 · 2018-09-11T19:31:53.793

135

In case you came here looking for a way to modify logging of any (possibly deeply nested) module, use logging.Logger.manager.loggerDict to get a dictionary of all of the logger objects. The returned names can then be used as the argument to logging.getLogger:

import requests
import logging
for key in logging.Logger.manager.loggerDict:
    print(key)
# requests.packages.urllib3.connectionpool
# requests.packages.urllib3.util
# requests.packages
# requests.packages.urllib3
# requests.packages.urllib3.util.retry
# PYREADLINE
# requests
# requests.packages.urllib3.poolmanager

logging.getLogger('requests').setLevel(logging.CRITICAL)
# Could also use the dictionary directly:
# logging.Logger.manager.loggerDict['requests'].setLevel(logging.CRITICAL)

Per user136036 in a comment, be aware that this method only shows you the loggers that exist at the time you run the above snippet. If, for example, a module creates a new logger when you instantiate a class, then you must put this snippet after creating the class in order to print its name.

edited Sep 11 '18 at 19:31

answered Mar 24 '16 at 19:37

kbrose

1,562
1
12
12

3

Thank you, this helped me silence `urllib3` log messages when using `boto3`. The logger in such case is `botocore.vendored.requests.packages.urllib3`, so I used this: `logging.getLogger("botocore.vendored.requests.packages.urllib3").setLevel(logging.WARNING)` and I finally got rid of the messages. – Bob Dem Jul 31 '17 at 13:28
Many thanks for this! Altering the print criteria allowed me to isolate that python-elasticsearch was the culprit in my case. – Robert Townley Nov 22 '17 at 17:04
2

Be aware that this will not work when modules create their loggers inside their class you'd call later, like the `APScheduler` does when you call `BackgroundScheduler.BackgroundScheduler()`. – user136036 Feb 20 '18 at 02:21
1

@user136036: logger objects are singletons, it doesn't matter if you or the library get to create them first. If you use the exact same name as the library uses, *it will work*. – Martijn Pieters Nov 08 '19 at 15:53
2

I think they are saying that if you list the loggers before a library has created its logger, then it won’t be listed. Which is correct. – kbrose Nov 09 '19 at 00:05
This helped me find the module that was writing log messages with request information that I wanted to get rid off (`azure`). Thank you so much! – stinodego Dec 09 '21 at 18:23
This is a lifesaver. I was trying to disable logging by setting `propagate = False` and could not get it working no matter what. Finally, this helped me to get logger name as `urllib3.connectionpool` as `urllib3` and `connectionpool` would not work. – Shriraj Hegde Jul 14 '23 at 14:06

score 32 · Answer 3 · answered Feb 12 '14 at 10:55

32

import logging
urllib3_logger = logging.getLogger('urllib3')
urllib3_logger.setLevel(logging.CRITICAL)

In this way all the messages of level=INFO from urllib3 won't be present in the logfile.

So you can continue to use the level=INFO for your log messages...just modify this for the library you are using.

answered Feb 12 '14 at 10:55

shaolin

463
5
13

4

I suggest using `setLevel(logging.WARNING)` to log also possible warning and error messages. – Mikko Koho Oct 15 '15 at 15:34

score 26 · Answer 4 · answered Feb 28 '20 at 09:41

26

import logging

# Only show warnings
logging.getLogger("urllib3").setLevel(logging.WARNING)

# Disable all child loggers of urllib3, e.g. urllib3.connectionpool
logging.getLogger("urllib3").propagate = False

answered Feb 28 '20 at 09:41

Martin Thoma

124,992
159
614
958

score 19 · Answer 5 · answered Feb 01 '16 at 19:52

19

For anybody using logging.config.dictConfig you can alter the requests library log level in the dictionary like this:

'loggers': {
    '': {
        'handlers': ['file'],
        'level': level,
        'propagate': False
    },
    'requests.packages.urllib3': {
        'handlers': ['file'],
        'level': logging.WARNING
    }
}

answered Feb 01 '16 at 19:52

theherk

6,954
3
27
52

@SebastianWagner Django uses `dictConfig` under the hood. – uhbif19 Nov 12 '18 at 12:25
Thank you so much! This is pretty good. One place to rule all of the library logs!! :) – MehmedB Dec 10 '19 at 08:35

sorin · Answer 6 · 2013-04-17T14:27:53.163

15

Let me copy/paste the documentation section which it I wrote about week or two ago, after having a problem similar to yours:

import requests
import logging

# these two lines enable debugging at httplib level (requests->urllib3->httplib)
# you will see the REQUEST, including HEADERS and DATA, and RESPONSE with HEADERS but without DATA.
# the only thing missing will be the response.body which is not logged.
import httplib
httplib.HTTPConnection.debuglevel = 1

logging.basicConfig() # you need to initialize logging, otherwise you will not see anything from requests
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

requests.get('http://httpbin.org/headers')

edited Apr 17 '13 at 14:27

answered Apr 16 '13 at 17:08

sorin

161,544
178
535
806

What's the point of being more specific than just 'requests' though, from a practical POV? – aknuds1 Apr 16 '13 at 18:39
But what do you gain by calling logging.getLogger("requests.packages.urllib3") instead of logging.getLogger("requests"), considering that you want to affect the logging of the requests library? – aknuds1 Apr 17 '13 at 14:45
1

Do you mean that you wish to enable logging within requests.packages.urllib3? If so, you're answering the wrong question. – aknuds1 Apr 17 '13 at 14:47
@aknuds1 It's up to you if you want to disable or enable them, I only put the code that fully controls this :) – sorin Jun 24 '13 at 08:55
4

I think you've misunderstood the question's scope. – aknuds1 Jun 24 '13 at 15:27

score 8 · Answer 7 · answered Dec 06 '18 at 20:39

Setting the logger name as requests or requests.urllib3 did not work for me. I had to specify the exact logger name to change the logging level.

First See which loggers you have defined, to see which ones you want to remove

print(logging.Logger.manager.loggerDict)

And you will see something like this:

{...'urllib3.poolmanager': <logging.Logger object at 0x1070a6e10>, 'django.request': <logging.Logger object at 0x106d61290>, 'django.template': <logging.Logger object at 0x10630dcd0>, 'django.server': <logging.Logger object at 0x106dd6a50>, 'urllib3.connection': <logging.Logger object at 0x10710a350>,'urllib3.connectionpool': <logging.Logger object at 0x106e09690> ...}

Then configure the level for the exact logger:

   'loggers': {
    '': {
        'handlers': ['default'],
        'level': 'DEBUG',
        'propagate': True
    },
    'urllib3.connectionpool': {
        'handlers': ['default'],
        'level': 'WARNING',
        'propagate' : False
    },

I have them in Django settings, in base.py. Where to put them of course depends on your project setup. — Mikko, Jan 27 '20 at 17:15

score 5 · Answer 8 · answered Jul 09 '17 at 07:34

5

If You have configuration file, You can configure it.

Add urllib3 in loggers section:

[loggers]
keys = root, urllib3

Add logger_urllib3 section:

[logger_urllib3]
level = WARNING
handlers =
qualname = requests.packages.urllib3.connectionpool

answered Jul 09 '17 at 07:34

Sank

121
1
4

1

This is a perfectly valid answer for people who use configuration file. Not sure why it got so many down vote? – Patrick Sep 21 '18 at 06:25

score 4 · Answer 9 · answered Oct 22 '19 at 23:06

This answer is here: Python: how to suppress logging statements from third party libraries?

You can leave the default logging level for basicConfig, and then you set the DEBUG level when you get the logger for your module.

logging.basicConfig(format='%(asctime)s %(module)s %(filename)s:%(lineno)s - %(message)s')
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

logger.debug("my debug message")

score 0 · Answer 10 · edited Jun 30 '18 at 12:36

0

I'm not sure if the previous approaches have stopped working, but in any case, here's another way of removing the warnings:

PYTHONWARNINGS="ignore:Unverified HTTPS request" ./do-insecure-request.py

Basically, adding an environment variable in the context of the script execution.

From the documentation: https://urllib3.readthedocs.org/en/latest/security.html#disabling-warnings

edited Jun 30 '18 at 12:36

Derlin

9,572
2
32
53

answered Mar 10 '16 at 03:46

newlog

1,050
1
11
23

score 0 · Answer 11 · answered Nov 22 '17 at 17:03

Kbrose's guidance on finding which logger was generating log messages was immensely useful. For my Django project, I had to sort through 120 different loggers until I found that it was the elasticsearch Python library that was causing issues for me. As per the guidance in most of the questions, I disabled it by adding this to my loggers:

      ...
      'elasticsearch': {
          'handlers': ['console'],
          'level': logging.WARNING,
      },     
      ...

Posting here in case someone else is seeing the unhelpful log messages come through whenever they run an Elasticsearch query.

score 0 · Answer 12 · answered Sep 13 '22 at 18:24

0

In my case what helped me (python 3.7)

import http.client as http_client
http_client.HTTPConnection.debuglevel = 0

answered Sep 13 '22 at 18:24

Szymon Roziewski

956
2
20
36

score -1 · Answer 13 · answered Feb 09 '16 at 02:22

-1

simple: just add requests.packages.urllib3.disable_warnings() after import requests

answered Feb 09 '16 at 02:22

evandrix

6,041
4
27
38

2

I did not find this method in my version. Disabling warnings is excessive, as the annoying messages are level `INFO`. – tripleee Apr 28 '16 at 08:43

How do I disable log messages from the Requests library?

13 Answers13

Linked

Related