0

I have the following dataframe

------------------------------------------------------
|longitude  | latitude  | geomType |    geom          |
------------------------------------------------------    
|-7.0737816 |33.82666166|Polygon   |[GEOMETRY - 113 o]|
-------------------------------------------------------

I want to apply this query on this dataframe

I use the folowing code

dataframe= sparkSession.sql("select ST_GeomFromText('POINT("+col("longitude")+" , '',"+col("latitude")+")')");

I get this error

Exception in thread "main" org.apache.spark.sql.AnalysisException: Undefined function: 'ST_GeomFromText'. This function is neither a registered temporary function nor a permanent function registered in the database 'default'.

Note that I'm using java language on spark

I need your help .

Thank you

HBoulmi
  • 333
  • 5
  • 16
  • 1
    `ST_GeomFromText` is not defined, what did you expect? From where did you get that function? – Lino Jul 09 '20 at 12:10
  • 2
    I added the dependencies in pom.xml is it not enough? '''geospark-sql''' dependencies – HBoulmi Jul 09 '20 at 12:13
  • 2
    No, it's a Postgres extension. This function come from PostGIS, and it will not work if your database is not a geospatial database – Erwan Daniel Jul 09 '20 at 12:47
  • 2
    Does this answer your question? [How to enable Postgis Query in Spark SQL](https://stackoverflow.com/questions/48305560/how-to-enable-postgis-query-in-spark-sql) – Erwan Daniel Jul 09 '20 at 12:48
  • Does this answer your question? [ST\_geomfromtext function using Spark / java](https://stackoverflow.com/questions/62830434/st-geomfromtext-function-using-spark-java) – Som Jul 10 '20 at 17:13

1 Answers1

0

Since the ST_GeomFromText is not the part of org.apache.spark.sql.functions so it will not recognise it internally. You need to first define the UDF for this function. means you need to define the definition of that function and then register that function with spark as UDF then only you can use this function. You can use following syntax: sqlContext.udf.register("ST_GeomFromText", <name of function or definition of function>)

Thanks

code.gsoni
  • 695
  • 3
  • 12
  • 2
    I put for example a function which takes as parameter the two columns (longitude and latitude) of dataframe ? – HBoulmi Jul 09 '20 at 13:50