remove brackets, apostrophes

Question

I have a program that generates random ips and grabs banners but when i run python3 main.py :80/ there's no output.

main.py

import ipaddress, concurrent.futures, requests, sys, random

def ipv4():
    for _ in range(10000):
        ip = (ipaddress.IPv4Address(random.randint(0,2 ** 32)))
        IP_list = []
        IP_list.append(str(ip).strip('\n'))
    return IP_list

def scan(IP):
     headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36'}
     r = requests.get('http://' + IP + sys.argv[1], headers=headers, timeout=0.5)
     print(r.url, r.status_code, r.headers['Server'])


ipv4_list = ipv4()

with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
    executor.map(scan, ipv4_list)

test.py

import ipaddress
import random

def ipv4():
    for _ in range(1000):
        ip = (ipaddress.IPv4Address(random.randint(0,2 ** 32)))
        IP_list = []
        IP_list.append(str(ip).strip('\n'))
        print(IP_list)

ipv4()

I've checked the output of ipv4() function and the problem seems to be that it encloses ips in brackets and apostrophes. I also looked at this SO question for IP_list.append(str(ip).strip('\n')), but it doesn't seem to work.

Part of the problem is that you have: `IP_list = []; IP_list.append(...`. You are creating (and returning) a one element list. — quamrana, Aug 25 '20 at 09:55
Please update the indentation of your code. Python is very sensitive to indentation, as are python programmers. — quamrana, Aug 25 '20 at 09:57
One problem is that you only produce `ipv4_list` of length one. The other is that you don't catch the `ConnectTimeoutError` exceptions that `requests.get()` throws and `executor.map()` silently swallows them. — quamrana, Aug 25 '20 at 10:28
You are printing a list with strings. Python will display the list and the fact that it contains strings, using Python notation. That doesn't mean the values contain brackets and quotes. All you have are simple strings of IP addresses. You don't even need to strip them, there are no newlines being generated here. — Martijn Pieters, Aug 25 '20 at 10:40
You are, however, recreating the `IP_list` list each iteration in your loop. Move the `IP_list = []` statement **outside** of the `for` loop, as the first thing your `ipv4()` function executes. — Martijn Pieters, Aug 25 '20 at 10:41
Side note: you are generating random IPs without checking if the resulting address can even be reached. You want to skip private use and loopback IPs, for example. Use `if ip.is_global: IP_list.append(str(ip))` to only include IP addresses that are not reserved for special purposes. — Martijn Pieters, Aug 25 '20 at 10:53

score 2 · Accepted Answer · answered Aug 25 '20 at 10:46

Your problem doesn't come from apostrophes or brackets.

First, let's make some quality changes:

main.py

...
def ipv4():
    IP_list = []  # This should be outside the loop
    for _ in range(100):
        ip = ipaddress.IPv4Address(random.randint(0,2 ** 32))  # no need for extra parentheses
        IP_list.append(str(ip))  # no need for .strip
    return IP_list
...

Now, let's solve the problem. Take a look at this piece:

def scan(IP):
     headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36'}
     r = requests.get('http://' + IP + sys.argv[1], headers=headers, timeout=0.5)
     print(r.url, r.status_code, r.headers['Server'])

Your code makes more than a few assumptions, one of which is pretty huge (and is the cause for failing): that the request will be successful. Let's fix that:

from urllib3.exceptions import ConnectTimeoutError
from requests.exceptions import ConnectionError
...

def scan(IP):
     headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36'}
     try:
         r = requests.get('http://' + IP + ':80/', headers=headers, timeout=1.5)  # a more sensible timeout
     except (ConnectTimeoutError, ConnectionError):
         print(f"Connection to {IP} failed.")
         return
     print(r.url, r.status_code, r.headers['Server'])

Now, if the request is not successful you will get a helpful message.

What happened

When you do parallel programming the errors aren't always apparent. In this case the threads fail the execution of your function but don't show anything. Programming with threads in Python has its peculiarities and a good place to start is the docs and this answer.

remove brackets, apostrophes

1 Answers1