How to append data into python dict when key values are same?

Question

I'm trying to store some data into dict but the key are same for all values, i tried update() but update ignore if the key already present in the dict. Wold be great if someone explain how to append data with same key value!

Here is the code i'm trying

from bs4 import BeautifulSoup
import requests

data = {}
proxy_url = 'https://free-proxy-list.net/'
req = requests.get(proxy_url)
soup = BeautifulSoup(req.content,'lxml')
table = soup.findAll('table')[0].findAll('tr')
for i in table:
    ip = i.select('td:nth-of-type(1)')
    port = i.select('td:nth-of-type(2)')

    if ip:
        ipx = ip[0].text

    if port:
        portx = port[0].text
        proxy = ('http//'+ipx+':'+portx).encode('utf-8')
        data.update({'http':proxy})

print(data)

The output dict i want:

data = {
  'http': 'http://10.10.1.10:3128',
  'http': 'http://10.10.1.10:1080',
}

*key are same for all values* Then it is not a `dict`, not atleast in Python. — DirtyBit, Mar 21 '19 at 08:17
Dictionaries cannot have same keys, You could perhaps append a counter like `1` to keep the keys unqiue. — DirtyBit, Mar 21 '19 at 08:18
If everything is `http` and there's many of them, then a more suitable datastructure is a `list` - either of just the address or if you really wanted, a list of `dict`s. — Jon Clements, Mar 21 '19 at 08:19
By definition dictionaries have unique keys, else how will you even access them?? — anand_v.singh, Mar 21 '19 at 08:19
Why do you even want a dict for this? A list would be very much suitable! — DirtyBit, Mar 21 '19 at 08:21
Assuming that you want to use `proxies` for `requests`, each key can have only one value so you can't have multiple `'http'` proxies in your dict. But you could store the proxies in a list and select one for your requests. — t.m.adam, Mar 21 '19 at 08:26

Nitin Pandey · Answer 1 · 2019-03-21T08:27:30.223

A python dictionary will not have a key having multiple values. Instead, you might want to have a list to hold these set of values. For example, you could do something like this:

data = {
  'http': ['http://10.10.1.10:3128', 'http://10.10.1.10:1080']
}

Suppose you want to store even and odd numbers till 100. You could do :

output = {'even':[],'odd':[]}
for number in range(0,100):
    if number%2==0:
        output['even'].append(number)
    else:
        output['odd'].append(number)

score 1 · Answer 2 · answered Mar 21 '19 at 08:24

The two ways I see to it;

Use a list to store all the requesting urls
Use a dict with a key http that has a list of all the urls.

Hence:

from bs4 import BeautifulSoup
import requests

data = []     # to store the urls in a list
dict_ = {}    # to store the urls in a dict as a list
proxy_url = 'https://free-proxy-list.net/'
req = requests.get(proxy_url)
soup = BeautifulSoup(req.content,'lxml')
table = soup.findAll('table')[0].findAll('tr')
for i in table:
    ip = i.select('td:nth-of-type(1)')
    port = i.select('td:nth-of-type(2)')

    if ip:
        ipx = ip[0].text

    if port:
        portx = port[0].text
        proxy = ('http//'+ipx+':'+portx)
        data.append(proxy)

dict_.update({'http':data})

print("List of urls: {}".format(data))
print("Dict of urls: {}".format(dict_))

OUTPUT:

List of urls: ['http//223.25.101.242:59504', 'http//36.89.183.77:61612', . . .]
Dict of urls: {'http': ['http//223.25.101.242:59504', 'http//36.89.183.77:61612', . . .]}

How to append data into python dict when key values are same?

2 Answers2