transform touple with (key and list) to (key and list item)

Question

I have an RDD with (key, list[word1, word2, word3]) and i want to transform this to (key, word1), (key, word2)... (key, word-n), can anyone point me in the right direction on how to solve this?

Possible duplicate of [What is the difference between map and flatMap and a good use case for each?](https://stackoverflow.com/questions/22350722/what-is-the-difference-between-map-and-flatmap-and-a-good-use-case-for-each) — David, Apr 23 '18 at 13:01

score 1 · Answer 1 · edited Apr 23 '18 at 14:19

Use list comprehension:

key, list_ = ('key', ['word1', 'word2', 'word3'])
result = [(key, item) for item in list_]
print(result)

Output:

[('key', 'word1'), ('key', 'word2'), ('key', 'word3')]

You can apply this solution to your rdd using flatMap():

myrdd = sc.parallelize([('key', ['word1', 'word2', 'word3'])])
myrdd.flatMap(lambda row: [(row[0], item) for item in row[1]]).collect()
#[('key', 'word1'), ('key', 'word2'), ('key', 'word3')]

score 0 · Answer 2 · edited Apr 23 '18 at 14:19

Use a list comprehension, iterate through tuple and associate first element with every item in the second element:

>>> tupl = ('key', ['word1', 'word2', 'word3'])  
>>> [(tupl[0], tupl[1][i]) for i in range(len(tupl[1]))]
[('key', 'word1'), ('key', 'word2'), ('key', 'word3')]

You can apply this solution to your rdd using flatMap():

myrdd = sc.parallelize([('key', ['word1', 'word2', 'word3'])])
myrdd.flatMap(lambda tupl: [(tupl[0], tupl[1][i]) for i in range(len(tupl[1]))]).collect()
#[('key', 'word1'), ('key', 'word2'), ('key', 'word3')]

transform touple with (key and list) to (key and list item)

2 Answers2