4

I have 4 dimensions of data. In R, I'm using plot3d with the 4th dimension being color. I'd like to now use SVM to find the best regression line to give me the best correlation. Basically, a best fit hyperplane dependent on the color dimension. How can I do this?

CodeGuy
  • 28,427
  • 76
  • 200
  • 317
  • What is it that you are trying to regress? You haven't given a dependent variable. Are you clear on the meaning of a support vector machine? – Iterator Nov 04 '11 at 20:55
  • revised: dependent on the color dimension (the 4th dimension) – CodeGuy Nov 05 '11 at 00:40
  • That's helpful. Btw the usual terminology is that the color variable depends on the other variables. Better terminology is that the other variables are predictor variables and the color variable is the response variable. – Iterator Nov 05 '11 at 00:46
  • so i figured out how to get my svm fit object. now i'm just confused as to how to draw that hyperplane in a 3d plot. how can i plot a fit object? – CodeGuy Nov 05 '11 at 00:50
  • I'm away from my computer, but see "classifly" or rggobi for examples. – Iterator Nov 05 '11 at 01:40

2 Answers2

10

This is the basic idea (of course the specific formula will vary depending on your variable names and which is the dependent):

library(e1071)

data = data.frame(matrix(rnorm(100*4), nrow=100))

fit = svm(X1 ~ ., data=data)

Then you can use regular summary, plot, predict, etc. functions on the fit object. Note that with SVMs, the hyper-parameters usually need to be tuned for best results. you can do this with the tune wrapper. Also check out the caret package, which I think is great.

John Colby
  • 22,169
  • 4
  • 57
  • 69
  • can you give an example of how I could use the tune function. also, how would you suggest I use the caret? – CodeGuy Nov 04 '11 at 20:07
  • what's X1? my 4 dimensions are x,y,z,q – CodeGuy Nov 04 '11 at 20:10
  • @CodeGuy It's just an example. Sample code. `X1` is the name of the first column in the example he provided. – joran Nov 04 '11 at 20:25
  • @CodeGuy In particular, it's also the dependent variable in this model. Do you have a dependent variable? – Iterator Nov 04 '11 at 20:56
  • Also check out `?formula` for how to specify a formula. If you read `?svm`, though, you'll see that that's not the only way you can specify a model. For SVM- and tuning-specific stuff, the manuals and vignettes for the `e1071` and `caret` packages are very thorough. – John Colby Nov 04 '11 at 22:07
  • yeah, I'd like the regression to depend on the color variable. okay, so I can use that. now how do I get data from the fit such that I can overly the hyperplane. i printed the fit and i see things such as cost and gamma. but how can I get the equation of the line, so that I can plot the 3d hyperplane? – CodeGuy Nov 04 '11 at 22:46
  • With the `predict` generic function I reference above. This is same basic model-building workflow used throughout R, so I would recommend getting an intro to R book if the steps are unfamiliar. – John Colby Nov 04 '11 at 23:25
  • the predict will give me a function? can you show an example of plotting that as an hyperplane? that's the part i'm unsure of, not the steps. – CodeGuy Nov 04 '11 at 23:38
  • 1
    See this other Q/A I helped with: http://stackoverflow.com/questions/7390173/svm-equations-from-e1071-r-package – John Colby Nov 04 '11 at 23:46
  • that seems to be for divisioning data, not for regression. can you give an example of how to plot the hyperplane given the results of fit as in your code above? – CodeGuy Nov 05 '11 at 00:35
5

Take a look on svm function in the e1071 package. You can also consider the kernelab, klaR or svmpath packages.

EDIT: @CodeGuy, John has provided you an example. I suppose your 4 dimensions are features that you use to classify your data, and that you have also another another variable that is the real class. 

y  <- gl(4, 5)
x1 <- c(0,1,2,3)[y]
x2 <- c(0,5,10,15)[y]
x3 <- c(1,3,5,7)[y]
x4 <- c(0,0,3,3)[y]

d <- data.frame(y,x1,x2,x3,x4)

library(e1071)
svm01 <- svm(y ~ x1 + x2 + x3 + x4, data=d)
ftable(predict(svm01), y)  # Tells you how your svm performance
joran
  • 169,992
  • 32
  • 429
  • 468
Manuel Ramón
  • 2,490
  • 2
  • 18
  • 23
  • I have looked at it, and I can't figure it out. Can you give me example code. I have 4 dimensions of data. – CodeGuy Nov 04 '11 at 20:06
  • what do you mean another variable? imagine a 3D plot with data. then there is a 4th dimension, which is given by the color of the data points. i'd like to find the best regression hyperplane in that 3d plot that is dependent on the 4th dimension (the color). can you update the code to reflect this. how could I get the equation of the hyperplane so that I could actually graph it to see what it looks like? – CodeGuy Nov 04 '11 at 23:07
  • there is an error in the code you posted. did you try running it? – CodeGuy Nov 05 '11 at 00:36