0

It doesn't seem spark sql with hive supports "sql like". Here is the example code that I use for testing in spark-shell

case class Department(id: String, name: String)
val department1 = new Department("123456", "Computer Science")
val department2 = new Department("789012", "Mechanical Engineering")
val department3 = new Department("345678", "Theater and Drama")
val department4 = new Department("901234", "Indoor Recreation")
val dSeq = Seq(department1, department2, department3, department4)
val dfd = dSeq.toDF()
dfd.registerTempTable("departments")
val re=sqlContext.sql("select * from departments where name like '*Rec*'")
re.show()

It will show no result. and I check the sqlContext and it is hiveContext

scala> sqlContext
res15: org.apache.spark.sql.SQLContext =org.apache.spark.sql.hive.HiveContext@4f03729f

Does any one know why?

dbspace
  • 231
  • 1
  • 2
  • 10
  • It does work just fine. You're just using a wrong pattern. `LIKE` is using SQL simple regular expression where wildcard is `%` (`'%Rec%'`), not Java regular expressions. For full regex support use `RLIKE`. – zero323 Apr 30 '16 at 17:36
  • I want to use sql context. and yes '%Rec% works. I should have try this first. but some how in another post the other user is having issue with % and the suggestion is to use * like what I post and that it is why I post it because it is not working. – dbspace Apr 30 '16 at 18:55

0 Answers0