How to create unique combinations from specific fields in a dataframe?

Asked Jul 30 '19 at 16:16

Active Jul 31 '19 at 12:24

Viewed 44 times

I am trying to create all combinations of items from four fields in my dataframe. I'm testing the code below, and something is wrong because I keep getting 'out of memory' errors.

import itertools
A = df['ID']
B = df['PRICE']
C = df['YIELD']
D = df['SCORE']
a = [A, B, C, D]

data = list(itertools.product(*a))
idx = ['c{}'.format(i) for i in range(1, len(data)+1)]
df = pd.DataFrame(data, index=idx, columns=list('abc')).T

df

I have 10,000 rows of data in this example. In other dataframes there will be many more rows. Is this possible, or is the resulting object just too large?

edited Jul 31 '19 at 12:24

Georgy

12,464
7
65
73

asked Jul 30 '19 at 16:16

ASH

20,759
19
87
200

`((Number of columns)^2)*(Number of rows)` will be the number of rows you will get at the end. – Vishnudev Krishnadas Jul 30 '19 at 16:31
In the case of this 10000 row example, `len(data)` = 10000^4 – Trenton McKinney Jul 30 '19 at 18:05
You might try [itertools.combinations](https://docs.python.org/3/library/itertools.html#itertools.combinations) instead. – Trenton McKinney Jul 30 '19 at 18:12
Does this answer your question? [All combinations of a list of lists](https://stackoverflow.com/questions/798854/all-combinations-of-a-list-of-lists) – AMC Feb 08 '20 at 01:25

How to create unique combinations from specific fields in a dataframe?

0 Answers0