2

Due to the neuralnet package doesn't have ReLU function, so I try to write the code for ReLU function. But there is an error I don't understand. Please see my code and error information below.

relu<-function(x){ifelse(x>=0,x,0)}
nn <- neuralnet(y~a+b+c+d+e+f,data=train,hidden=c(5),linear.output=T,act.fct = relu)

Error in deriv.formula(eval(parse(text = text)), "x", func = eval(parse(text = text2)),: Function 'ifelse' is not in the derivatives table

Jeffrey
  • 41
  • 2
  • 8

1 Answers1

6

ReLU can be found in the sigmoid package. To add the activation function used needs to be a differential function.

Here are couple simple implementation of ReLU 

relu <- function(x) {x * (x>=0)}
relu <- function(x) {max(0,x)}
Carlos Santillan
  • 1,077
  • 7
  • 8