problem with silhouette score with affinity propagation

Question

I try to run this code but i have an error : ValueError: Number of labels is 1. Valid values are 2 to n_samples - 1 (inclusive). Furthermore, it is said that : Affinity propagation did not converge, this model will not have any cluster centers. What is the solution for these 2 problems

preferences = range(-15000,-5000,50) # arbitraty chosen range
af_sil_score = [] # silouette scores

for p in preferences:
    AF = AffinityPropagation(preference=p, max_iter=200).fit(X)
    no_of_clusters.append((len(np.unique(AF.labels_))))
    af_sil_score.append(silhouette_score(X, AF.labels_))
    
af_results = pd.DataFrame([preferences, no_of_clusters, af_sil_score], index=['preference','clusters', 'sil_score']).T
af_results.sort_values(by='sil_score', ascending=False).head() # display only 5 best scores```

score 0 · Answer 1 · answered Aug 28 '22 at 03:14

For some of your values of preference , it is not suitable for your dataset, and will not converge (giving you the convergence error). When the algorithm fails, it does not return labels and it doesn't make sense to apply silhouette_score (your second error).

You can check this page or other post to see how to set the preference value. Or you can simply try a range of values, and simply return -1 when the number of clusters is below 2.

Using an example :

from sklearn.cluster import AffinityPropagation
from sklearn.metrics import silhouette_score
from sklearn.datasets import make_blobs
import pandas as pd

X, labels_true = make_blobs( n_samples=250, random_state=0, n_features =5, centers = 3 )

preferences = range(-1000,-100,100) 
af_sil_score = [] # silouette scores
no_of_clusters = []

for p in preferences:

    AF = AffinityPropagation(preference=p, max_iter=200).fit(X)
    no_of_clusters.append(len(AF.cluster_centers_))
    if len(AF.cluster_centers_) > 1:
        af_sil_score.append(silhouette_score(X, AF.labels_))
    else :
        af_sil_score.append(-1)

Results look like this, you can see it starts to work around -500 in the example dataset above :

pd.DataFrame({'preferences':preferences,'no_of_clusters':no_of_clusters,'sil':af_sil_score})

   preferences  no_of_clusters       sil
0        -1000               0 -1.000000
1         -900               0 -1.000000
2         -800               0 -1.000000
3         -700               0 -1.000000
4         -600               0 -1.000000
5         -500               3  0.716733
6         -400               3  0.716733
7         -300               3  0.716733
8         -200               3  0.716733

problem with silhouette score with affinity propagation

1 Answers1