Reading a file sequentially using multiple threads in python

Asked Jul 10 '18 at 19:22

Active Jul 10 '18 at 19:49

Viewed 36 times

I'm trying to create a script that takes a path of a text file then goes through each line and sends a request then return the status code of the response and i'm trying to implement threads but the problem is that 50 threads for example sends 50 requests to each line not 50 requests to 50 different lines so how can i implement that?

import requests
import os
import threading
import sys


threads = []


def checkalive():
    txt_file = open("%s" % (sys.argv[1]), "r")

    headers = {
        'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0',
        }

    for line in file:
        requests.get("https://%s/" % (line), headers=headers)

for i in range(50):
    t = threading.Thread(target=checkalive)
    threads.append(t)
    t.start()

edited Jul 10 '18 at 19:49

nijm

2,158
12
28

asked Jul 10 '18 at 19:22

Muhammed Azmy

You could first read the lines split them up and then start the threads? – nijm Jul 10 '18 at 19:39
Thanks for your reply, i didnt get what you mean, can you please explain more? – Muhammed Azmy Jul 10 '18 at 19:44
Possible duplicate of [Python: interdependent process/thread queues](https://stackoverflow.com/questions/27515167/python-interdependent-process-thread-queues) – ivan_pozdeev Jul 10 '18 at 20:32

Reading a file sequentially using multiple threads in python

0 Answers0