Create data partition into training, testing and validation - split in R

Question

I wanted to split my training data in to 70% training, 15% testing and 15% validation. I am using the createDataPartition() function of the caret package. I am splitting it like the following

train <- read.csv("Train.csv")
test <- read.csv("Test.csv")

split=0.70
trainIndex <- createDataPartition(train$age, p=split, list=FALSE)
data_train <- train[ trainIndex,]
data_test <- train[-trainIndex,]

Is there any way of splitting into training, testing and validation using createDataPartition() like the following H2o approach?

data.hex <- h2o.importFile("Train.csv")
splits <- h2o.splitFrame(data.hex, c(0.7,0.15), destination_frames = c("train","valid","test"))
train.hex <- splits[[1]]
valid.hex <- splits[[2]]
test.hex  <- splits[[3]]

Do two splits: (1) original data into training and temp and (2) temp into testing and validation. — Gregor Thomas, Apr 07 '16 at 16:54
Duplicate of: http://stackoverflow.com/questions/36068963/r-how-to-split-a-data-frame-into-training-validation-and-test-sets — stackoverflowuser2010, Apr 07 '16 at 18:32
August 2017: there's now the `rsample` package in R https://topepo.github.io/rsample/ — alexpghayes, Aug 08 '17 at 19:03

score 9 · Answer 1 · edited Sep 24 '17 at 03:29

9

A method using the sample() function in base R is

splitSample <- sample(1:3, size=nrow(data.hex), prob=c(0.7,0.15,0.15), replace = TRUE)
train.hex <- data.hex[splitSample==1,]
valid.hex <- data.hex[splitSample==2,]
test.hex <- data.hex[splitSample==3,]

edited Sep 24 '17 at 03:29

Erin LeDell

8,704
1
19
35

answered Apr 07 '16 at 17:06

lmo

37,904
9
56
69

`> nrow(data.hex) [1] 25192 > nrow(train.hex) [1] 8398 > valid.hex <- data.hex[splitSample==2,] > nrow(valid.hex) [1] 8397 > test.hex<- data.hex[splitSample==3,] > nrow(test.hex) [1] 8397` but the difference between them is only 1. is this correct? – Mahsolid Apr 07 '16 at 17:39
1

Oops. Forgot the size argument. – lmo Apr 07 '16 at 17:46
2

Note that this is (quasi) random, so the sizes will be approximately equal to 0.7, 0.15, 0.15, but not exactly. For replication purposes, you would want to set the seed above the first line: `set.seed(some integer)` – lmo Apr 07 '16 at 17:53

score 0 · Answer 2 · answered Oct 25 '20 at 07:12

0

Take a look at train,validation, test split model in CARET in R. The idea is to use createDataPartition() twice. First time p=0.7 to create 70% train and 30% remaining data. Second time p=0.5 on remaining data to create 15% testing and 15% validate.

answered Oct 25 '20 at 07:12

Perceptron

399
3
11

Create data partition into training, testing and validation - split in R

2 Answers2