2

I am training a neural network for kaggle's handwitten digit compitition. To practice R, I translated Andrew Ng's neural network octave program into R. So I need to find a substitute for fmincg.

I found optimx and played with many methods in it. I observed that fmincg was faster and more accurate than optimx.

Here's my test:

Let's minimize y = x^4, without constraint. Everyone knows x should equal to zero.

Here's my matlab program:

clc;
clear all;
close all;
options = optimset('MaxIter', 50);
init = 1:.1:50;
init = init';
newFunc = @(p) costFunc(p);
tic
[nn_params, cost] = fmincg(newFunc, init, options);
toc

with costFunc function:

function [cost grad] = costFunc(x)
  cost = sum(x.^4);
  grad = 4*x.^3;
end

And the R script:

costFunc = function (x)
{
  return(sum(x^4))
}

grad = function(x) {
  return(4*x^3)
}

startTime = proc.time()
results = optimx(seq(1, 50,.1), fn=costFunc, gr=grad,
                 itnmax=50, method=c("Rcgmin"))
proc.time() - startTime

Octave finished in 0.064 seconds with accuracy to 10^-8. R finished in 1.8 seconds with accuracy only to 10^-4.

Of course this problem is far too easy. When I trained neural networks, for some calculations, octave can give results in minutes but R seems running forever.

I am wondering if anyone has come across this issue and who can suggest better minimization function in R as the substitute of fmincg in octave?

Fei
  • 85
  • 7
  • From `help(Rcgmin)`: "This code is entirely in R to allow users to explore and understand the method." Find a C implementation you can use (or write one) if you need better efficiency. Regarding accuracy: Try adjusting the `tol` parameter in the `control` settings. – Roland Dec 17 '14 at 12:15
  • `optimx` gives the same result?　Are you sure you are not comparing chalk and cheese? http://stackoverflow.com/questions/10770934/matlabs-fminsearch-different-from-octaves-fmincg – Khashaa Dec 17 '14 at 12:37
  • @Roland Looks like I need to quit R for doing the neural network ml. It runs too slowly. – Fei Dec 17 '14 at 21:13
  • @Khashaa I am comparing fmincg on octave and optimx with rcgmin or other methods on R. – Fei Dec 17 '14 at 21:21
  • 1
    I don't understand your comment. Looking at the source code it should be quite easy to implement this with Rcpp. – Roland Dec 18 '14 at 08:06
  • @Roland Thanks a lot! I didn't know mapping C++ equivalents is so easy. – Fei Dec 22 '14 at 23:02
  • @Roland Any facillity to map Fortran equivalents. Just curious. – Fei Dec 22 '14 at 23:03
  • Calling Fortran code from R shouldn't be a problem. After all, significant parts of R are written in Fortran. But since I don't speak Fortran I don't know details. – Roland Dec 24 '14 at 14:58

0 Answers0