Subsetting columns and counting the 1's (TURF analysis?)

Question

The aim is to count the corresponding 1's in the rows of each subset (>2) of columns:

    0   2   4
0   0   1   0
1   1   1   1
2   1   0   0
3   1   1   0
4   1   0   0
... ... ... ...

In above example we would have 4 subsets. Then the idea is to summarize these counts in a bar plot where each bar is labelled according the subset.

The aim is to make an UpSet plot

mozway · Accepted Answer · 2022-12-09T15:49:14.957

1

It looks like you're looking for an UpSetPlot:

# pip install upsetplot
import upsetplot

upsetplot.plot(df.astype(bool).value_counts())

Output:

With all combinations

upsetplot.plot(df.astype(bool).value_counts()
                 .reindex(product([True, False], repeat=3), fill_value=0)
              )

older answer

It looks like you might want something like:

df.value_counts().plot.bar()

Output:

Or, by column name for 1 values:

(df.reset_index().melt('index', var_name='cols')
   .query('value == 1')
   .groupby('index')['cols'].agg(frozenset)
   .value_counts().plot.bar()
)

Output:

edited Dec 09 '22 at 15:49

answered Dec 09 '22 at 14:06

mozway

194,879
13
39
75

Second plot is close, but subset needs to be >2 and is missing subsets 2,4 and 0,4 – Sean_TBI_Research Dec 09 '22 at 15:38
There is no case with 2/4, so you want 0? – mozway Dec 09 '22 at 15:39
The intersection size is not yet correct, for example, [0, 4] has one row in common, so intersection size is 1. [0, 2, 4] has intersection size 1. – Sean_TBI_Research Dec 09 '22 at 16:35
1

Please provide a clear example, with a mockup graph, what you want is currently unclear. On my graph, there is a count of 1 for 0,2,4 – mozway Dec 09 '22 at 16:40
Is there a way to customize the plot? Increase figure size, add colors, etc.? – Sean_TBI_Research Dec 09 '22 at 16:49
Yes of course, you can do anything you want. The underlying plot is made with matplotlib. I suggest you to read the documentation, give it a try and open a new question if needed. – mozway Dec 09 '22 at 17:16
do you know how to add punctuation to the numbers? If I set show_counts=True – Sean_TBI_Research Dec 10 '22 at 16:30
Not sure you can do this natively, but your can [modify the tick labels after plotting](https://stackoverflow.com/questions/11244514/modify-tick-label-text) – mozway Dec 10 '22 at 17:00
Most matplotlib statements do not work with UpSet: 'UpSet' object has no attribute 'style' – Sean_TBI_Research Dec 11 '22 at 08:31
@Sean you should use `dic = upsetplot.plot(df.astype(bool).value_counts())`, then each value of the `dic` is a [`matplotlib.Axes`](https://matplotlib.org/stable/api/axes_api.html) object that you can modify. – mozway Dec 11 '22 at 18:18
What was the purpose of that reindex and product? – Sean_TBI_Research Dec 11 '22 at 19:58
First run `product([True, False], repeat=3)` to see what it outputs (all combinations of True/False for 3 items). `reindex` ensures all combinations are present in the DataFrame's index. – mozway Dec 11 '22 at 20:01
I've tried everything, but can't seem to be able to add .2f style to the numbers. Any ideas? – Sean_TBI_Research Dec 13 '22 at 11:16
@Sean_TBI_Research can you open a new question with a minimal self-sufficient code to replicate your issue? – mozway Dec 13 '22 at 11:37
I've just posted a new question – Sean_TBI_Research Dec 14 '22 at 16:52
Have you had the time to take a look at the question I posted? – Sean_TBI_Research Dec 15 '22 at 13:25

Subsetting columns and counting the 1's (TURF analysis?)

1 Answers1

older answer