Stack Overflow
Stack Overflow

Questions tagged [geospark]

GeoSpark is a cluster computing system for processing large-scale spatial data. GeoSpark extends Apache Spark / SparkSQL with a set of out-of-the-box Spatial Resilient Distributed Datasets (SRDDs)/ SpatialSQL that efficiently load, process, and analyze large-scale spatial data across machines.

45 questions
12
votes
2 answers

TypeError: 'JavaPackage' object is not callable (spark._jvm)

I'm setting up GeoSpark Python and after installing all the pre-requisites, I'm running the very basic code examples to test it. from pyspark.sql import SparkSession from geo_pyspark.register import GeoSparkRegistrator spark =…
Jessica Chambers
  • 1,246
  • 5
  • 28
  • 56
4
votes
2 answers

sedona error : java.lang.NoClassDefFoundError: org/opengis/referencing/FactoryException

/usr/share/spark-3.0/bin/pyspark --queue=szsc --master=yarn --packages…
Liangliang Qiu
  • 41
  • 3
4
votes
1 answer

Spark task fails to write rows into ORC table

I run the following code for a spatial join on geometry fields: val coverage = DimCoverageReader.apply(spark, params) coverage.createOrReplaceTempView("dim_coverage") val uniqueGeometries = spark.table(params.UniqueGeometriesTable) …
Maria
  • 195
  • 1
  • 11
3
votes
0 answers

Py4JJavaError- NotClassFound error while using GeoSpark

I am using GeoSpark (Apache Sedona) library to read geospatial data from .csv file. I want to read polygon from this csv file, but getting the below error: : java.lang.NoClassDefFoundError:…
Mohd Tajuddin Arshad
  • 71
  • 3
3
votes
0 answers

Error while testing the Sedona VIZ example

I'm trying to run the Sedona Spark Visualization tutorial code. Example link: https://sedona.apache.org/tutorial/viz/ The following dataset is my…
Joao Marcos
  • 31
  • 3
3
votes
1 answer

Read a binary column in spark using java language

I have a DataFrame witch contains a Binary column Type. DataFrame…
melissa maya
  • 119
  • 10
2
votes
1 answer

Best approach for geospatial indexes in Palantir Foundry

What the recommended approach is for building a pipeline that needs to find a point contained in a polygon (shape) in Planatir Foundry? In the past, this has been pretty difficult in Spark. GeoSpark has been pretty popular, but can still lag. If…
codestrap
  • 43
  • 5
2
votes
2 answers

Apache-Sedona with Pyspark - java.lang.ClassCastException: [B cannot be cast to org.apache.spark.unsafe.types.UTF8String

I am loading a data from parquet in spark in which one of the column is of Binary Type. I want to convert this column to geometry type for this I am using Apache Sedona/GeoSpark. I am using st_geomfromwkb for this but getting error df =…
Mohd Tajuddin Arshad
  • 71
  • 3
2
votes
1 answer

Unable to configure GeoSpark in Spark Session :

I have been trying to configure geospark with Spark Session so that I can use spatial application on PySpark. I followed this link & tried to run the code which is mentioned as follows. try: import pyspark from pyspark import SparkContext,…
user2538041
  • 61
  • 1
  • 7
2
votes
0 answers

GeoSpark functions using java language

I have a dataframe with a binary column type…
HBoulmi
  • 333
  • 5
  • 16
2
votes
1 answer

SQL query and dataframes using Spark/java

I am a beginner in spark and I got stuck in how to make a sql request using dataframe. I have the two followings dataframe. dataframe_1 +-----------------+-----------------+----------------------+---------------------+ |id …
HBoulmi
  • 333
  • 5
  • 16
1
vote
0 answers

Using sparklyr and geospark in R, convert Spark DataFrame to Spark Spatial DataFrame (for point to line distance calculation)

I have a Spark DataFrame with latitude and longitude where I'm trying to calculate the distance between the coordinates and a polyline. The dataframe I'm working with is large (about 10 billion observations) and I load using spark_read_parquet(), so…
Rob Marty
  • 378
  • 1
  • 9
1
vote
1 answer

unable to register spark session on GeoSparkRegistrator.registerAll(spark)

I have been trying to register spark with GeoSpark. I have installed apache sedona 3.1.3 version in python 3.7. Spark session has created using #Import required libraries import os import folium import geopandas as gpd from pyspark.sql import…
Narayan Das
  • 13
  • 3
1
vote
2 answers

Why is Apache Sedona not reading this Shapefile properly?

I'm using Apache Spark v3.0.1 and Apache Sedona v1.1.1 and I'm trying to read a Shapefile into a SpatialRDD. I first tried the example provided by the Sedona library (more specifically, the code inside testShapefileConstructor method), and it just…
PiFace
  • 526
  • 3
  • 19
1
vote
0 answers

How to use sedona spark sql or geo spatail in spark

Hi have the below polygon data(basically of 500 meter) like Location_N,latitude,longitude,geometry Dohrighat (NP),26.262341,83.512787,"POLYGON ((83.51780100000001 26.262341, 83.517791 26.262059, 83.51776200000001 26.261777, 83.517712 26.261498,…
san san
  • 118
  • 2
  • 13
1
2 3