i'm trying to make a SQL structured streaming from a MQTT Broker:
test = spark.readStream.format("org.apache.bahir.sql.streaming.mqtt.MQTTStreamSourceProvider")\
.option("clientId", "experiment")\
.option("brokerUrl","tcp://<broker>:1883")\
.option("topic",'#')\
.option('QoS',0)\
.option('connectionTimeout',0)\
.option('keepAlive',5)\
.option('autoReconnect',True)\
.option('persistence','memory')\
.load()
These commands looks fine, but when i try to write the stream i get an error:
PARQUET_PATH="/tmp"
test.writeStream.format("parquet")\
.option('checkpointLocation', PARQUET_PATH + "/_chk")\
.start(PARQUET_PATH)
java.lang.AbstractMethodError: org.apache.bahir.sql.streaming.mqtt.MQTTStreamSource.planInputPartitions()Ljava/util/List;
Did someone got an error like this? I already tried everything and did not get past this.
I'm using Spark 2.4.0, Scala 2.11 and org.apache.bahir:spark-sql-streaming-mqtt_2.11:2.4.0-SNAPSHOT
