partially define initial centroid for scikit-learn K-Means clustering

Question

Scikit documentation states that:

Method for initialization:

‘k-means++’ : selects initial cluster centers for k-mean clustering in a smart way to speed up convergence. See section Notes in k_init for more details.

If an ndarray is passed, it should be of shape (n_clusters, n_features) and gives the initial centers.

My data has 10 (predicted) clusters and 7 features. However, I would like to pass array of 10 by 6 shape, i.e. I want 6 dimensions of centroid of be predefined by me, but 7th dimension to be iterated freely using k-mean++.(In another word, I do not want to specify initial centroid, but rather control 6 dimension and only leave one dimension to vary for initial cluster)

I tried to pass 10x6 dimension, in hope it would work, but it just throw up the error.

Gabriel M · Accepted Answer · 2018-10-30T08:52:07.463

Sklearn does not allow you to perform this kind of fine operations.

The only possibility is to provide a 7th feature value that is random, or similar to what Kmeans++ would have achieved.

So basically you can estimate a good value for this as follows:

import numpy as np
from sklearn.cluster import KMeans

nb_clust = 10
# your data
X = np.random.randn(7*1000).reshape( (1000,7) )   

# your 6col centroids  
cent_6cols = np.random.randn(6*nb_clust).reshape( (nb_clust,6) ) 

# artificially fix your centroids
km = KMeans( n_clusters=10 )
km.cluster_centers_ = cent_6cols

# find the points laying on each cluster given your initialization
initial_prediction = km.predict(X[:,0:6])

# For the 7th column you'll provide the average value 
# of the points laying on the cluster given by your partial centroids    
cent_7cols = np.zeros( (nb_clust,7) )
cent_7cols[:,0:6] = cent_6cols
for i in range(nb_clust):
    init_7th = X[ np.where( initial_prediction == i ), 6].mean()
    cent_7cols[i,6] =  init_7th

# now you have initialized the 7th column with a Kmeans ++ alike 
# So now you can use the cent_7cols as your centroids
truekm = KMeans( n_clusters=10, init=cent_7cols )

I am getting an ValueError: Incorrect number of features. Got 7 features, expected 6 > initial_prediction = km.predict(X[0:6]) > X = self._check_test_data(X) > n_features, expected_n_features)) I also checked X.shape (7455, 7) and cent_6cols.shape (10, 6). — Martan, Oct 30 '18 at 06:09
That's why you have to slice X before passing it to predict. I correct the code, slicing should be done as follows `initial_prediction = km.predict(X[:,0:6])` — Gabriel M, Oct 30 '18 at 08:52

score 1 · Answer 2 · answered Oct 30 '18 at 18:36

That is a very nonstandard variation of k-means. So you cannot expect sklearn to be prepared for every exotic variation. That would make sklearn slower for everybody else.

In fact, your approach is more like certain regression approaches (predicting the last value of the cluster centers) rather than clustering. I also doubt the results will be much better than simply setting the last value to the average of all points assigned to the cluster center using the other 6 dimensions only. Try partitioning your data based on the nearest center (ignoring the last column) and then setting the last column to be the arithmetic mean of the assigned data.

However, sklearn is open source.

So get the source code, and modify k-means. Initialize the last component randomly, and while running k-means only update the last column. It's easy to modify it this way - but it's very hard to design an efficient API to allow such customizations through trivial parameters - use the source code to customize at his level.

partially define initial centroid for scikit-learn K-Means clustering

2 Answers2

Linked