0

I am trying to run the apache beam program using Python SDK on my local machine. I created and activated the python virtual environment and also installed apache beam using pip.

However, when I trigger the code using the below command, it gives the attribute error and says:

module filename.py has no attribute __path__

Below is the command I ran (by going to the venv folder):

python -m filename.py

Please help me out on this.. I am trying to learn apache beam

import apache_beam as beam
p=beam.Pipeline()
lines= p | beam.io.ReadFromText('path\\My_sample_input_file.txt');

lines | beam.io.WriteToText('path\\output2.txt')
RobC
  • 22,977
  • 20
  • 73
  • 80
surya
  • 3
  • 1

1 Answers1

1

You shouldn't use the -m flag to run a script. See What is the purpose of the -m switch?

To run a python script:

python myfile.py

To import and run a module:

python -m myfile

This will also work because the current working directory is in the search path for python modules.

oLas
  • 1,171
  • 1
  • 9
  • 17
  • I tried what you said. it did not give an error this time, however, it did not perform what is expected of it either..I was trying to print the lines read from source file and this is what it displayed..Pcollection[ReadFromText/Read.None].. I am using python 3 with beam.. – surya Aug 14 '19 at 16:37
  • Your problem is more likely to be answered as a new question. Remember to provide a minimum working example to reproduce your problem. You can mark this question as resolved if it solved your original problem. – oLas Aug 14 '19 at 19:45
  • Thanks it solved my original problem.. but my new problem is it is neither writing to a new file nor giving error...The code snippet and everything is same as above except that I removed -m option.. – surya Aug 15 '19 at 17:25
  • you must run the pipeline with `p.run()`. You can learn the basics of Apache Beam by following the WordCount example in the docs: https://beam.apache.org/get-started/wordcount-example/ – oLas Aug 15 '19 at 22:17