0

How we can read a csv from REST API and process it in java spark,

Basically I am looking for java equivalent code of below scala code

process-csv-from-rest-api-into-spark

import scala.io.Source._
import org.apache.spark.sql.{Dataset, SparkSession}

var res = fromURL(url).mkString.stripMargin.lines.toList
val csvData: Dataset[String] = spark.sparkContext.parallelize(res).toDS()

val frame = spark.read.option("header", true).option("inferSchema",true).csv(csvData)
frame.printSchema()
mck
  • 40,932
  • 13
  • 35
  • 50
Debug Logs
  • 61
  • 6

1 Answers1

0

We can use approach mentioned in below link to read in data and add each line in list.

This list can be passed to parallelise()

Java read CSV file from the web

Debug Logs
  • 61
  • 6