How to split a dataset, which has two associated files, into training and test datasets by Python?

Asked Oct 01 '17 at 22:48

Active Oct 01 '17 at 22:48

Viewed 95 times

For example I have two files, sentences.txt and labels.txt
sentences.txt:

This is a sentence
This is not a sentence
Hello world
Hello world too

labels.txt:

Each number in labels.txt is associated with the sentence in sentences.txt in the same position. I would like to randomly select 50% for testing and 50% for training, and it seems that I cannot simply random.shuffle(data) like in this question. So how could I split the dataset randomly? Thanks.

asked Oct 01 '17 at 22:48

user8142520

What is the size of your data? – cs95 Oct 01 '17 at 22:51
down vote accepted If you want to split the data set once in two halves, you can use numpy.random.shuffle, or numpy.random.permutation if you need to keep track of the indices – Fady Saad Oct 01 '17 at 22:52
The assumption here is that if you are working with a classifier, you're using either sklearn/numpy/pandas. – cs95 Oct 01 '17 at 22:53
data size is about 10k lines – user8142520 Oct 01 '17 at 23:00

How to split a dataset, which has two associated files, into training and test datasets by Python?

0 Answers0