1

When uploading a dataframe as csv from pyspark to GCS, how do I overwrite an existing file? If I were to run this code twice, I would get the error below

test_row = Row(id=1)
test_df = spark.createDataFrame([test_row])

#saving the spark dataframe to GCS
p="test_file"
bucket='test_bucket'
table_id = 'test_folder'
q="gs://{}/{}/{}".format(bucket,table_id,p)
test_df.write.csv(q)

'path gs://test_bucket/test_folder/test_file already exists.;'

user147529
  • 555
  • 1
  • 7
  • 18

0 Answers0