-1

BODS job is creating CSV Files.

IS there a way to convert CSV Files to Parquet and Upload to S3 Bucket in SAP BODS.

The Current approach i am using is below for Converting the CSV to Parquet

  1. Create a CSV File in the Folder that BODS is accessable.

  2. Created a Python Script and placed the Script in the Package Folder and below is the code import os,sys os.chdir("/usr/sap/DBO/dataservices/DataQuality/python/lib/python3.7/site-packages") sys.path.append('/usr/sap/DBO/dataservices/DataQuality/python/lib/python3.7/site-packages') import pandas as pd df = pd.read_csv('/ds_ext_share/BODS_DEV/Output/xxxx.csv') df.to_parquet('/ds_ext_share/BODS_DEV/Output/xxxx.parquet')

  3. I am calling the above script as Exec in my BODS Job in a script

exec('/usr/sap/DBO/dataservices/DataQuality/python/lib/python3.7/site-packages','XXXX.py' , '8');

The above code is not working need help in fixing the issue

Thanks Madhu

The CSV file is not converting to Parquet, need help in the script

Sandra Rossi
  • 11,934
  • 5
  • 22
  • 48
Madhu Reddy
  • 1
  • 1

1 Answers1

-1

Removing BODS from the equation the question remains how to convert a CSV file to parquet in python Pandas and PyArrow! This has been asked and answered in a similar thread here.

jmuiruri
  • 11
  • 3
  • Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Dec 26 '22 at 02:08