28

Usually when I do dendrograms and heatmaps, I use a distance matrix and do a bunch of SciPy stuff. I want to try out Seaborn but Seaborn wants my data in rectangular form (rows=samples, cols=attributes, not a distance matrix)?

I essentially want to use seaborn as the backend to compute my dendrogram and tack it on to my heatmap. Is this possible? If not, can this be a feature in the future.

Maybe there are parameters I can adjust so it can take a distance matrix instead of a rectangular matrix?

Here's the usage:

seaborn.clustermap¶
seaborn.clustermap(data, pivot_kws=None, method='average', metric='euclidean',
 z_score=None, standard_scale=None, figsize=None, cbar_kws=None, row_cluster=True,
 col_cluster=True, row_linkage=None, col_linkage=None, row_colors=None,
 col_colors=None, mask=None, **kwargs)

My code below:

from sklearn.datasets import load_iris
iris = load_iris()
X, y = iris.data, iris.target
DF = pd.DataFrame(X, index = ["iris_%d" % (i) for i in range(X.shape[0])], columns = iris.feature_names)

enter image description here

I don't think my method is correct below because I'm giving it a precomputed distance matrix and NOT a rectangular data matrix as it requests. There's no examples of how to use a correlation/distance matrix with clustermap but there is for https://stanford.edu/~mwaskom/software/seaborn/examples/network_correlations.html but the ordering is not clustered w/ the plain sns.heatmap func. 

DF_corr = DF.T.corr()
DF_dism = 1 - DF_corr
sns.clustermap(DF_dism)

enter image description here

O.rka
  • 29,847
  • 68
  • 194
  • 309
  • I'm not sure I understand the question. Is the second matrix not square? – mwaskom Aug 03 '16 at 23:49
  • Yea, the 2nd one is definitely square but it's b/c I fed it a distance matrix ( 1- correlation) while `sns.cluster_map` requires the rectangular data matrix. So basically it took my redundant square distance matrix, treated them as raw values, and then did linkage from that. Does that work mathematically? It doesn't seem to make sense since the input requires a rectangular data matrix and I think certain steps are being repeated. – O.rka Aug 04 '16 at 17:12
  • I think you need to edit the question to make it more clear what you want to know. As written you are asking how to make a square matrix, and you are showing a plot that is a square matrix. – mwaskom Aug 04 '16 at 17:46
  • K, I'll clear that up right now. – O.rka Aug 04 '16 at 18:04

1 Answers1

34

You can pass the precomputed distance matrix as linkage to clustermap():

import pandas as pd, seaborn as sns
import scipy.spatial as sp, scipy.cluster.hierarchy as hc
from sklearn.datasets import load_iris
sns.set(font="monospace")

iris = load_iris()
X, y = iris.data, iris.target
DF = pd.DataFrame(X, index = ["iris_%d" % (i) for i in range(X.shape[0])], columns = iris.feature_names)

DF_corr = DF.T.corr()
DF_dism = 1 - DF_corr   # distance matrix
linkage = hc.linkage(sp.distance.squareform(DF_dism), method='average')
sns.clustermap(DF_dism, row_linkage=linkage, col_linkage=linkage)

For clustermap(distance_matrix) (i.e., without linkage passed), the linkage is calculated internally based on pairwise distances of the rows and columns in the distance matrix (see note below for full details) instead of using the elements of the distance matrix directly (the correct solution). As a result, the output is somewhat different from the one in the question: clustermap

Note: if no row_linkage is passed to clustermap(), the row linkage is determined internally by considering each row a "point" (observation) and calculating the pairwise distances between the points. So the row dendrogram reflects row similarity. Analogous for col_linkage, where each column is considered a point. This explanation should likely be added to the docs. Here the docs's first example modified to make the internal linkage calculation explicit:

import seaborn as sns; sns.set()
import scipy.spatial as sp, scipy.cluster.hierarchy as hc
flights = sns.load_dataset("flights")
flights = flights.pivot("month", "year", "passengers")
row_linkage, col_linkage = (hc.linkage(sp.distance.pdist(x), method='average')
  for x in (flights.values, flights.values.T))
g = sns.clustermap(flights, row_linkage=row_linkage, col_linkage=col_linkage) 
  # note: this produces the same plot as "sns.clustermap(flights)", where
  #  clustermap() calculates the row and column linkages internally
Ulrich Stern
  • 10,761
  • 5
  • 55
  • 76