2

There is an API localhost:8000/api/postdatetime/, which is responsible for changing the online status of the driver. Every time a driver hits the API and if the response sent back then the driver status will be online otherwise the driver status will be offline.

Drivers need to give the response within every 10 seconds. If there is no response from the driver then he/she will be marked as offline automatically.

Currently what I am doing is getting all the logged in drivers and hitting the above-mentioned API one driver at a time.

For the simulation purpose, I populated thousands of drivers. To maintain the online-offline of thousands of drivers using my approach will leave almost many drivers offline.

The code for my approach is as described below:

online-offline.py

import requests
import random
import math
from rest_framework.authtoken.models import Token

from ** import get_logged_in_driver_ids
from ** import Driver


def post_date_time():
    url = 'localhost:8000/api/postdatetime/'

    while True:
        # getting all logged in driver ids 
        logged_in_driver_ids = get_logged_in_driver_ids() 

        # filtering the driver who are logged in and storing their user id in the driver_query variable
        driver_query = Driver.objects.only('id', 'user_id').filter(id__in=logged_in_driver_ids).values('user_id')

        # storing the user id of driver in list
        driver_user_ids = [driver['user_id'] for driver in driver_query]

        # getting number of drivers who are supposed to hit the API
        drivers_subset_count = math.ceil(len(driver_user_ids)*(random.randint(75, 85)/100))

        # shuffle the driver ids list to randomize the driver id order
        random.shuffle(driver_user_ids)

        # getting the driver list of length drivers_subset_count
        required_drivers = driver_user_ids[:drivers_subset_count]

        for driver in required_drivers:
            token = Token.objects.only('key', 'user_id').get(user_id=driver)
            req_headers = {
                'Content-Type': 'application/json',
                'Accept': 'application/json',
                'Authorization': 'Token' + ' ' + str(token)
            }
            response = requests.post(url=url, headers=req_headers)
            print(response.text)



if __name == '__main__':
    post_date_time()

Is there any idea that I can post request the localhost:8000/api/postdatetime/ API asynchronously and can handle about 2-3000 of drivers within 10 seconds?

I am really new at implementing async codes. I have read some articles on the aiohttp library but got confused at the time of implementing it.

This online-offline.py will run for the whole time of the simulation.

Any help will be highly appreciated. Thank you :)

dipesh
  • 763
  • 2
  • 9
  • 27
  • 1
    If I understand situation correctly and API is local, asynchronous approach won't help you. Please, read [this answer](https://stackoverflow.com/a/51180506/1113207) for detailed explanation. – Mikhail Gerasimov Nov 04 '19 at 19:37
  • Thank you very much for your reply and the explanation. But what if I used the non-local API, and want to achieve the result. How can be this achieved? Or, What if I used the local APIs and want to achieve the simulation. How can be this possible? All I want to do is handle the 2-3000 API requests within the 10 seconds for an unlimited time frame. – dipesh Nov 05 '19 at 04:27

1 Answers1

1

In case of non-local API asyncio can help you. To make requests asynchronously you have to:

  • use special syntax (async def, await - read here for details)
  • make requests non-blocking way (so you can await them)
  • use asyncio.gather() of similar way to make multiple requests parallely"
  • start event loop

While it is possible to make requests library work with asyncio using threads it's easier and better to use already asyncio-compatible library like aiohttp.

Take a look at code snippets here *: they contain examples of making multiple concurrent requests. Rewrite you code same way, it'll look like:

import asyncio


async def driver_request(driver):
    # ...
    # use aiohttp to make request with custom headers:
    # https://docs.aiohttp.org/en/stable/client_advanced.html#custom-request-headers


async def post_date_time():
    # ...

    await asyncio.gather(*[
        driver_request(driver) 
        for driver 
        in required_drivers
    ])


asyncio.run(post_date_time())

Locally you won't see any effect by default, so to test it locally you have to make localhost send response with delay to emulate real-life network delay.

* In latest Python version event loop can be run with just asyncio.run()

Mikhail Gerasimov
  • 36,989
  • 16
  • 116
  • 159