Using APCluster on Pre-Classified Data - Coloring Dendograms and Nicely Formatted Output

Question

Question 1: I am trying to work with the plot() function on an AggExResult object and the clusters in the documentation (https://cran.r-project.org/web/packages/apcluster/apcluster.pdf) work as expected.

In my own data, I have an additional column in the input which provides a pre-defined “target” for classification purposes, and I am wondering if there is a way to have the dendogram labels highlighted by color (e.g. red=class 0, blue=class 1) with the class of the targets being factors (or characters). I am ultimately trying to visually display how many clusters contain "pure" vs. "mixed" classes. Here is some slightly modified code from the online documentation to show roughly what my input data looks like:

cl1Targ <- matrix(nrow=50,ncol=1)
for(c1t in 1:nrow(cl1Targ)){ cl1Targ[c1t]  <- as.factor(0) }
cl2Targ <- matrix(nrow=50,ncol=1)
for(c2t in 1:nrow(cl2Targ)){ cl2Targ[c2t]  <- as.factor(1) }

## create two Gaussian clouds
#cl1 <- cbind(rnorm(50,0.2,0.05),rnorm(50,0.8,0.06))
#cl2 <- cbind(rnorm(50,0.7,0.08),rnorm(50,0.3,0.05))
cl1 <- cbind(rnorm(50,0.2,0.05),rnorm(50,0.8,0.06),cl1Targ)
cl2 <- cbind(rnorm(50,0.7,0.08),rnorm(50,0.3,0.05),cl2Targ)
x <- rbind(cl1,cl2)
colnames(x) <- c('Column 1','Column 2','Class_ID')

## compute similarity matrix (negative squared Euclidean)
sim <- negDistMat(x, r=2)
## run affinity propagation
apres <- apcluster(sim, q=0.7)
## compute agglomerative clustering from scratch
aggres1 <- aggExCluster(sim)
## plot dendrogram
plot(aggres1, main='aggres1 w/ target') # 

How would I color the dendogram by the target defined in the input?

Question 2: When I show() the example data’s APResult, I see the following:

show(apres)     
APResult object

Number of samples     =  100
Number of iterations  =  165
Input preference      =  -0.01281384
Sum of similarities   =  -0.1222309
Sum of preferences    =  -0.1409522
Net similarity        =  -0.2631832
Number of clusters    =  11

Exemplars:
   8 17 24 37 43 52 58 68 92 95 99
Clusters:
   Cluster 1, exemplar 8:
      7 8 9 25 31 36 39 42 47 48
   Cluster 2, exemplar 17:
      6 11 13 15 17 18 19 23 32 35
   Cluster 3, exemplar 24:
      2 5 10 24 45

When I use my own data, I see the following (the row.names, which are the drugs being clustered by gene expression mean fold change values)

show(apclr2q05_mean)

APResult object

Number of samples     =  1045
Number of iterations  =  429
Input preference      =  -390.0822
Sum of similarities   =  -89326.99
Sum of preferences    =  -83477.58
Net similarity        =  -172804.6
Number of clusters    =  214

Exemplars:
   amantadine_58mg6h_fc amiodarone_147mg3d_fc clarithromycin_56mg1d_fc fluconazole_394mg5d_fc ketoconazole_114mg5d_fc ketoconazole_2274mg1d_fc
   pantoprazole_1100mg1d_fc pantoprazole_1100mg3d_fc quetiapine_500mg5d_fc roxithromycin_312mg5d_fc torsemide_3mg3d_fc acetazolamide_250mg3d_fc
Clusters:
   Cluster 1, exemplar amantadine_58mg6h_fc:
      amantadine_58mg6h_fc promazine_100mg1d_fc cyproteroneAcetate_2500mg6h_fc danazol_2g5d_fc ivermectin_7500ug1d_fc letrozole_250mg6h_fc
      mefenamicAcid_93mg3d_fc olanzapine_23mg1d_fc secobarbital_20mg6h_fc zaleplon_100mg3d_fc
   Cluster 2, exemplar amiodarone_147mg3d_fc:
      amiodarone_147mg3d_fc amiodarone_147mg5d_fc aspirin_375mg5d_fc betaNapthoflavone_80mg5d_fc clofibrate_130mg3d_fc finasteride_800mg5d_fc
   Cluster 3, exemplar clarithromycin_56mg1d_fc:
      ciprofloxacin_72mg5d_fc ciprofloxacin_450mg6h_fc clarithromycin_56mg1d_fc clarithromycin_56mg3d_fc clarithromycin_56mg5d_fc
   Cluster 4, exemplar fluconazole_394mg5d_fc:
      fluconazole_394mg5d_fc

Also what I would expect in terms of content but I would like to format this for reporting purposes. I have tried to export this using dput() but I get a lot of extra unnecessary information in the output file. I am wondering how I might be able to export the same type of information from above along with the object name and target classifier mentioned above into a table that would look like the following (and add the name of the object to the output):

Name of object        =  apclr2q05_mean
Number of samples     =  1045
Number of iterations  =  429
Input preference      =  -390.0822
Sum of similarities   =  -89326.99
Sum of preferences    =  -83477.58
Net similarity        =  -172804.6
Number of clusters    =  214

Exemplars:                    Target
    amantadine_58mg6h_fc       1
    amiodarone_147mg3d_fc      1
    clarithromycin_56mg1d_fc   1
    fluconazole_394mg5d_fc     0
    ketoconazole_114mg5d_fc    0
    ketoconazole_2274mg1d_fc   0

Clusters:
   Cluster 1, exemplar amantadine_58mg6h_fc:
     Drug                            Target
     amantadine_58mg6h_fc            1
     promazine_100mg1d_fc            1
     cyproteroneAcetate_2500mg6h_fc  1
     danazol_2g5d_fc                 0
     ivermectin_7500ug1d_fc          0

   Cluster 2, exemplar amiodarone_147mg3d_fc:
     Drug                            Target
     Etc…

A big THANK YOU to Ulrich for his quick response to these questions by email and we wanted to share our discussion with the community so I will let him respond with his solution so that he gets the credit he deserves :-)

---

As an update, I tried to implement the answer to Question 1 and the sample code works as expected, but I am having trouble getting this to work on my data. The input data has two parts. The first is a matrix with the numeric measurement data including column and row labels:

> fci[1:3,1:3]
                      M30596_PROBE1 AI231309_PROBE1 NM_012489_PROBE1
amantadine_58mg1d_fc     0.05630744     -0.10441722       0.41873201
amantadine_58mg6h_fc    -0.42780274     -0.26222322       0.02703001
amantadine_220mg1d_fc    0.35260779     -0.09902214       0.04067055

The second is the "target" values in Factor format, each of which corresponds to same row in fci above:

> targs[1:3]
 amantadine_58mg1d_fc  amantadine_58mg6h_fc amantadine_220mg1d_fc 
                    0                     0                     0 
Levels: 0 1

From here, the tree was built as below:

# build the AggExResult:
aglomr1 <- aggExCluster(negDistMat(r=2), fci)

# convert the data
tree <- as.dendrogram(aglomr1)

# assign the color codes
colorCodes <- c("0"="red", "1"="green")
names(targs)  <- rownames(fci)
xColor <- colorCodes[as.character(targs)]
names(xColor) <- rownames(fci)

# plot the colored tree
labels_colors(tree) <- xColor[order.dendrogram(tree)]
plot(tree, main="Colored Tree")

The tree was generated but the leaves were not colored. Doing some digging:

> head(xColor)
    0     0     0     0     0     0 
"red" "red" "red" "red" "red" "red" 

That part seems to work as expected in terms of the targets having the correct colors assigned, but the rownames are not in xColor, and the line labels_colors(tree) <- xColor[order.dendrogram(tree)] does not return similar labels, but rather what appear to be row numbers, or NAs:

> head(order.dendrogram(tree))
[1] "295" "929" "488" "493" "233" "235"

> head(labels_colors(tree))
295 929 488 493 233 235 

> head(xColor[order.dendrogram(tree)])
<NA> <NA> <NA> <NA> <NA> <NA> 
 NA   NA   NA   NA   NA   NA 

How would I get the line labels_colors(tree) <- xColor[order.dendrogram(tree)] to behave in the same way as the example provided? Specifically, what I am trying to show is the leaf lables such as amantadine_58mg1d_fc being highlighted in the color that corresponds to the target (0/1).

Answer 1

Here is my answer to your Question 1: the plot() method for 'AggExResult' objects internally uses the plot.dendrogram() method. Since this method does not allow for coloring leaves of dendrograms, this will not work. However, there is the 'dendextend' package which offers such a functionality. (BTW, I found that solution in another thread: Label and color leaf dendrogram in r) Since 'apcluster' offers some casts to 'hclust' and 'dendrogram' objects, this package's functionality can be used more or less directly.

So, here is some sample code:

library(apcluster)

## create two Gaussian clouds along with class labels 0/1
cl1 <- cbind(rnorm(50, 0.2, 0.05), rnorm(50, 0.8, 0.06))
cl2 <- cbind(rnorm(50, 0.7, 0.08), rnorm(50, 0.3, 0.05))
x <- cbind(Columns=data.frame(rbind(cl1, cl2)),
           "Class_ID"=factor(as.character(c(rep(0, 50), rep(1, 50)))))

## compute similarity matrix (negative squared Euclidean)
sim <- negDistMat(x[, 1:2], r=2)

## compute agglomerative clustering from scratch
aggres1 <- aggExCluster(sim)

## load 'dendextend' package
## install.packages("dendextend") ## if not yet installed
library(dendextend)

## convert object
tree <- as.dendrogram(aggres1)

## assign color codes
colorCodes <- c("0"="red", "1"="green")
xColor <- colorCodes[x$Class_ID]
names(xColor) <- rownames(x)

## plot color-labeled tree
labels_colors(tree) <- xColor[order.dendrogram(tree)]
plot(tree)

Answer 2

Here is my answer to your Question 2: Sorry, no such functionality is implemented in the 'apcluster' package. And since this is quite a special request, I am reluctant to include it the package (let alone the fact that show() methods cannot have additional arguments). So, alternatively, I want to provide you with a custom function that allows for labeling/grouping exemplars and samples:

library(apcluster)

## create two Gaussian clouds along with class labels 0/1
cl1 <- cbind(rnorm(50, 0.2, 0.05), rnorm(50, 0.8, 0.06))
cl2 <- cbind(rnorm(50, 0.7, 0.08), rnorm(50, 0.3, 0.05))
x <- cbind(Columns=data.frame(rbind(cl1, cl2)),
           "Class_ID"=factor(as.character(c(rep(0, 50), rep(1, 50)))))

## compute similarity matrix (negative squared Euclidean)
sim <- negDistMat(x[, 1:2], r=2)

## special show() function with labeled data
show.ExClust.labeled <- function(object, labels=NULL)
{
    if (!is(object, "ExClust"))
        stop("'object' is not of class 'ExClust'")

    if (is.null(labels))
    {
        show(object)
        return(invisible(NULL))
    }

    cat("\n", class(object), " object\n", sep="")

    if (!is.finite(object@l) || !is.finite(object@it))
        stop("object is not result of an affinity propagation run; ",
             "it is pointless to create 'APResult' objects yourself.")

    cat("\nNumber of samples     = ", object@l, "\n")
    if (length(object@sel) > 0)
    {
        cat("Number of sel samples = ", length(object@sel),
            paste("   (", round(100*length(object@sel)/object@l,1),
                  "%)\n", sep=""))
        cat("Number of sweeps      = ", object@sweeps, "\n")
    }
    cat("Number of iterations  = ", object@it, "\n")
    cat("Input preference      = ", object@p, "\n")
    cat("Sum of similarities   = ", object@dpsim, "\n")
    cat("Sum of preferences    = ", object@expref, "\n")
    cat("Net similarity        = ", object@netsim, "\n")
    cat("Number of clusters    = ", length(object@exemplars), "\n\n")

    if (length(object@exemplars) > 0)
    {
        if (length(names(object@exemplars)) == 0)
        {
            cat("Exemplars:\n")
            df <- data.frame("Sample"=object@exemplars,
                             Label=labels[object@exemplars])
            print(df, row.names=FALSE)

            for (i in 1:length(object@exemplars))
            {
                cat("\nCluster ", i, ", exemplar ",
                    object@exemplars[i], ":\n", sep="")

                df <- data.frame(Sample=object@clusters[[i]],
                                 Label=labels[object@clusters[[i]]])
                print(df, row.names=FALSE)
            }
        }
        else
        {
            df <- data.frame("Exemplars"=names(object@exemplars),
                             Label=labels[names(object@exemplars)])
            print(df, row.names=FALSE)

            for (i in 1:length(object@exemplars))
            {
                cat("\nCluster ", i, ", exemplar ",
                    names(object@exemplars)[i], ":\n", sep="")

                df <- data.frame(Sample=names(object@clusters[[i]]),
                               Label=labels[names(object@clusters[[i]])])
                print(df, row.names=FALSE)
            }
        }
    }
    else
    {
        cat("No clusters identified.\n")
    }
}


## create label vector (with proper names)
label <- x$Class_ID
names(label) <- rownames(x)

## run apcluster()
apres <- apcluster(sim, q=0.3)

## show with labels
show.ExClust.labeled(apres, label)