3

I'm using the module hcluster to calculate a dendrogram from a distance matrix. My distance matrix is an array of arrays generated like this:

import hcluster
import numpy as np

mols = (..a list of molecules)
distMatrix = np.zeros((10, 10))
  for i in range(0,10):       
    for j in range(0,10):
      sim = OETanimoto(mols[i],mols[j]) # a function to calculate similarity between molecules
      distMatrix[i][j] = 1 - sim

I then use the command distVec = hcluster.squareform(distMatrix) to convert the matrix into a condensed vector and calculate the linkage matrix with vecLink = hcluster.linkage(distVec).

All this works fine but if I calculate the linkage matrix using the distance matrix and not the condensed vector matLink = hcluster.linkage(distMatrix) I get a different linkage matrix (the distances between the nodes are a lot larger and topology is slightly different)

Now I'm not sure whether this is because hcluster only works with condensed vectors or whether I'm making mistakes on the way there.

Thanks for your help!

Mo Sander
  • 1,303
  • 1
  • 10
  • 14

1 Answers1

2

I knocked up a quick random example similar to yours and experienced the same problem. In the docstring it does say :

Performs hierarchical/agglomerative clustering on the condensed distance matrix y. y must be a :math:{n \choose 2} sized vector where n is the number of original observations paired in the distance matrix.

However, having had a quick look at the code, it seems like the intent is for it to both work with vector shaped and matrix shaped code: In hierachy.py there is a switch based upon the shape of the matrix. It seems however that the key bit of info is in the function linkage's docstring:

   - Q : ndarray
       A condensed or redundant distance matrix. A condensed
       distance matrix is a flat array containing the upper
       triangular of the distance matrix. This is the form that
       ``pdist`` returns. Alternatively, a collection of
       :math:`m` observation vectors in n dimensions may be passed as
       a :math:`m` by :math:`n` array.

So I think that the interface doesn't allow the passing of a distance matrix. Instead it thinks you are passing it m observation vectors in n dimensions . Hence the difference in result?

Does that seem reasonable?

Else just take a look at the code itself I'm sure you'll be able to debug it and figure out why your examples are different.

Cheers Matt

Jollyplum
  • 36
  • 1
  • Hi Matt,thanks a lot for your reply. It is reassuring to hear that passing of a vector is the way to go. – Mo Sander Apr 20 '11 at 14:37