0

I am storing RDD into mutable Hashmap using some key as below.

var datasets = new HashMap[String, RDD[T]]()
val feedRdd: RDD[T] = ...
datasets.put("someKey", feedRdd)

Now I am trying to fetch the same rdd from Hashmap and it is returning Option[RDD[T]] as below

val feedRddNew = datasets.get("someKey")

And it is giving error something like this.

Expression of type Option[RDD[T]] doesn't confirm to expected type RDD[T]

Basically I want to store rdd into Hashmap so that I can fetch it from Hashmap as I needed. Any thoughts on this? Please let me know if something is wrong or any alternate way. Thanks!

Anil Savaliya
  • 129
  • 1
  • 1
  • 6

1 Answers1

1
val feedRddNew = datasets.get("someKey")

This getter returns an Option[T] where T is the type stored in the map.

So, either None, or Some(T)

so you can do

val theActualValue = feedRddNew.get

or, you can just use

datasets("someKey")

which doesnt return an option (it just throws if the key is not found)

Erix
  • 7,059
  • 2
  • 35
  • 61