Spark & Scala: “negative select” columns in spark's org.apache.spark.sql.row

Asked Jun 17 '16 at 13:10

Active Jun 19 '16 at 18:47

Viewed 356 times

Similar to this question (which was solved using drop()) I have the same problem for the spark.sql.row. I checked the documentation, but wasn't able to figure it out on my own. I want to avoid a workaround like toSeq() if there's a cleaner alternative.

How do I retrieve all elements but a one given by index?

import org.apache.spark.sql._
val row = Row(1, 1.2, 57.12, null)
val colIndexToBeExluded = 1
// how do I get all (3) columns but colIndexToBeExluded ?

Thanks!

edited May 23 '17 at 11:58

Community

asked Jun 17 '16 at 13:10

Boern

7,233
5
55
86

2

What do you mean by "get ... columns"? What do you want to end up with? An array? Another Row? – The Archetypal Paul Jun 17 '16 at 14:45
I concur with @TheArchetypalPaul . I'm not sure that I understand your question. – eliasah Jun 17 '16 at 16:14
a `org.apache.spark.mllib.linalg.Vector` would be the optimum output since I want to apply machine learning on the remaining columns. – Boern Jun 18 '16 at 11:31
Not sure I understand how that would work, since Vectors hold Doubles and your Row doesn't have any (and only one numeric value) – The Archetypal Paul Jun 18 '16 at 11:40
Changed question accordingly - thanks. I do have only numbers ideed. – Boern Jun 19 '16 at 18:47

Spark & Scala: “negative select” columns in spark's org.apache.spark.sql.row

0 Answers0