how to pass headers to async with session.get()

Question

I would like to know how to pass the header in the following get call

headers = {
'User-Agent': 'Mozilla'
}
async def fetch(url, session):
async with session.get(url) as response:
    resp = await response.read()
    return resp

I tried the following but not getting any response.

headers = {
'User-Agent': 'Mozilla'
}
async def fetch(url, session):
async with session.get(url, headers=headers) as response:
    resp = await response.read()
    return resp

The objective is to call different urls in asynchronous mode. Need to know if there is any other alternate way as well but in any case, would need to pass the headers to get proper response.

Do you have reason to believe that the headers were _not_ correctly sent? If so, how did you conclude that? — user4815162342, Feb 19 '21 at 10:23
@user4815162342 I tried with normal "requests.get(url, headers=headers)" in the same code which gave me the response but when it comes to this line of code it is not. — Akhil Pakkath, Feb 19 '21 at 10:27
There might be other things that differ between `requests` and `aiohttp` unrelated to `User-Agent`. Before assuming that `headers=headers` doesn't work you should check whether the `User-Agent` is being sent. For example, you could make request to a server that just logs the headers received, etc. — user4815162342, Feb 19 '21 at 10:35

score 6 · Accepted Answer · answered Feb 19 '21 at 10:37

You can use httpbin.org for requests to see how servers sees your request:

import asyncio
import aiohttp
from pprint import pprint


headers = {
    'User-Agent': 'Mozilla'
}


async def fetch(url, session):
    async with session.get(url, headers=headers) as response:
        res = await response.json()
        pprint(res)


async def main():
    async with aiohttp.ClientSession() as session:
        await fetch("http://httpbin.org/get", session)


asyncio.run(main())

Result:

{'args': {},
 'headers': {'Accept': '*/*',
             'Accept-Encoding': 'gzip, deflate',
             'Host': 'httpbin.org',
             'User-Agent': 'Mozilla',
             'X-Amzn-Trace-Id': 'Root=1-602f94a7-3aa49d8c48ea04345380c67b'},
 'origin': '92.100.218.123',
 'url': 'http://httpbin.org/get'}

As you see 'User-Agent': 'Mozilla' was sent.

how to pass headers to async with session.get()

1 Answers1