Need help with example to understand what is DSL in Spark DataFram
"DataFrame allows developers to impose a structure onto a distributed collection of data, allowing higher-level abstraction; it provides a domain specific language API to manipulate your distributed data"
In the above definition what is Domain specific language API to manipulate your distributed data ? What does it mean. Please explain with some code example
It's hard to say for sure without the context, but I think that the author means that:
The DataFrame has an API (a set of functions that you can call on it) that provides a way of working with the data inside that is a higher level abstraction that working with the data direction.
For example to perform a "group by" (like in SQL), you can store the data in an array, and loop over it with a for-loop, while keeping a dictionary with the grouping keys and corresponding values. Or you can directly call "groupBy" on the DataFrame, which hides the complexity from you - it "abstracts it" by allowing you to use the more abstract concept of grouping, compared to the more concrete concept of looping over the array with the data.
Such abstractions are sometimes called a DSL, because you can view the function calls as a programming language itself that is specific to this type of problem/domain, meaning, data in a DataFrame.
