As a newbie, I created a pipeline with a couple of Transform functions, after having input from a file, it turns the code to lowercase. When I passed that outcome to the next stage it does not appear as a single string but multiple characters. Below is my code:
import apache_beam as beam
from apache_beam.options.pipeline_options import PipelineOptions
class ToLower(beam.DoFn):
def process(self, element):
#return [{'Data': element.lower()}]
return element.lower()
class ToReverse(beam.DoFn):
def process(self, element):
print(element)
return element
if __name__ == '__main__':
in_file = 'news.txt'
options = PipelineOptions()
with beam.Pipeline(options=PipelineOptions()) as p:
r = (
p | beam.io.ReadFromText(in_file)
| beam.ParDo(ToLower())
| beam.ParDo(ToReverse())
)
Assuming the content of news.txt is below:
Coronavirus cases in Pakistan doubled in one day with total tally at 106 on Monday
When I run the above code it prints the following instead of the reverse:
c
o
r
o
n
a
v
i
r
u
s
c
a
s
e
s
i
n
p
a
k
i
s
t
a
n
d
o
u
b
l
e
d
i
n
o
n
e
d
a
y
w
i
t
h
t
o
t
a
l
t
a
l
l
y
a
t
1
0
6
o
n
m
o
n
d
a
y
And when I change the return in ToLower to return [{'Data': element.lower()}] then it returns as a single string line. What is going on here?
