How to select items from a list based on probability

Question

I have lists a and b

a = [0.1, 0.3, 0.1, 0.2, 0.1, 0.1, 0.1]

b = [apple, gun, pizza, sword, pasta, chicken, elephant]

Now I want to create a new list c of 3 items

the 3 items are chosen form list b based on the probabilities in list a

the items should not repeat in list c

for example- output I am looking for

c = [gun,sword,pizza]

or

c = [apple, pizza, pasta]

note (sum of all values of list a is 1,number of items in list a and b is the same, actually i have a thousand items in both list a and b and i want to select hundred items from the list based on probability assigned to them,python3 )

Also, I'm not sure what this has to do with genetic algorithms or `fuzzywuzzy` so I'm removing those tags. — roganjosh, Jul 14 '18 at 08:49
I'm not sure you need Numpy, just [`random.choices()`](https://docs.python.org/3/library/random.html#random.choices). — Delgan, Jul 14 '18 at 08:54
Just realised the dupe I've suggested uses both python and numpy so it's covered — roganjosh, Jul 14 '18 at 08:54
@Delgan the docs suggest that it uses replacement and there doesn't seem to be an argument to avoid it? — roganjosh, Jul 14 '18 at 08:56
https://stackoverflow.com/questions/3679694/a-weighted-version-of-random-choice this worked perfectly — gokul gupta, Jul 14 '18 at 09:13

nosklo · Accepted Answer · 2018-07-14T09:24:38.113

7

Use random.choices:

>>> import random
>>> print(random.choices(
...     ['apple', 'gun', 'pizza', 'sword', 'pasta', 'chicken', 'elephant'], 
...     [0.1, 0.3, 0.1, 0.2, 0.1, 0.1, 0.1],
...     k=3
... ))
['gun', 'pasta', 'sword']

Edit: To avoid replacement, you can remove the selected item from the population:

def choices_no_replacement(population, weights, k=1):
    population = list(population)
    weigths = list(weights)    
    result = []
    for n in range(k):
        pos = random.choices(
            range(len(population)), 
            weights,
            k=1
        )[0]
        result.append(population[pos])
        del population[pos], weights[pos]
    return result

Testing:

>>> print(choices_no_replacement(
...     ['apple', 'gun', 'pizza', 'sword', 'pasta', 'chicken', 'elephant'],
...     [0.1, 0.3, 0.1, 0.2, 0.1, 0.1, 0.1],
...     k=3
... ))
['gun', 'pizza', 'sword']

edited Jul 14 '18 at 09:24

answered Jul 14 '18 at 08:55

nosklo

217,122
57
293
297

1

The documentation suggests that this picks with replacement and I can't see a way to avoid that. – roganjosh Jul 14 '18 at 08:56
@roganjosh added a way to do it – nosklo Jul 14 '18 at 09:20
I think it's just easier to use the numpy approach :) My "can't see a way to avoid that" was more in reference to `choices` not having an argument to prevent replacement. It seems some way behind the numpy method, but it's getting there :) Maybe Python 3.8 will fix it. – roganjosh Jul 14 '18 at 09:22

score 0 · Answer 2 · answered Mar 24 '22 at 12:14

In case you don't want to define in advance how many items you want to pick (so, you don't do something like k=3) and you just have probabilities, you can do as below. Note that your probabilities do not need to add up to 1 as shown, they can be independent of each other. Admittedly, I am not addressing your issue of preventing possible repetitions:

a = [0.2, 0.3, 0.9, 0.1]
b = ['apple', 'gun', 'pizza', 'sword'] 

selected_items = [item for p,item in zip(a,b) if random.random()<p]
print(selected_items)
>>>['apple','pizza']

How to select items from a list based on probability

2 Answers2