Python sklearn Value Error Target Variable

Question

I am running following code:

from sklearn.model_selection import train_test_split

X_train,X_test, y_train, y_test=train_test_split(X,y,stratify=y,test_size=0.3)

Output:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-27-b5740f8ae579> in <module>()
      1 from sklearn.model_selection import train_test_split
      2 
----> 3 X_train,X_test, y_train, y_test=train_test_split(X,y,stratify=y,test_size=0.3)

/Applications/anaconda3/lib/python3.6/site-packages/sklearn/model_selection/_split.py in train_test_split(*arrays, **options)
   2054                      random_state=random_state)
   2055 
-> 2056         train, test = next(cv.split(X=arrays[0], y=stratify))
   2057 
   2058     return list(chain.from_iterable((safe_indexing(a, train),

/Applications/anaconda3/lib/python3.6/site-packages/sklearn/model_selection/_split.py in split(self, X, y, groups)
   1202         """
   1203         X, y, groups = indexable(X, y, groups)
-> 1204         for train, test in self._iter_indices(X, y, groups):
   1205             yield train, test
   1206 

/Applications/anaconda3/lib/python3.6/site-packages/sklearn/model_selection/_split.py in _iter_indices(self, X, y, groups)
   1544         class_counts = np.bincount(y_indices)
   1545         if np.min(class_counts) < 2:
-> 1546             raise ValueError("The least populated class in y has only 1"
   1547                              " member, which is too few. The minimum"
   1548                              " number of groups for any class cannot"

ValueError: The least populated class in y has only 1 member, which is too few. The minimum number of groups for any class cannot be less than 2.

When I am running the exact same set of lines for another Machine Learning Project with different data, it works fine. What am I doing wrong?

Additional info related to the shape of the dataframe under consideration:

print(data.shape)
print(X.shape)
print(y.shape)

Output:

(3047, 33)
(3047, 32)
(3047, 1)

score 1 · Accepted Answer · answered Mar 08 '18 at 04:31

Since you are using stratify, number of samples belonging to each class need to be proportional in train and test. But you have a class in your data which have only a single sample. So either that can be in train or test at a time and this breaks the stratify option. Hence the error.

See my other answer here which describes a similar situation with example.

Python sklearn Value Error Target Variable

1 Answers1