1

I have a PySpark dataframe with a date field and I'm exporting this dataframe to a PostgreSQL database using the JDBC driver. I dataframe has types

df.dtypes
dt: timestamp
value: double
indi: integer

and before writing to PostgreSQL I order the dataframe by date:

df = df.orderBy('dt')
df.write.jdbc(postgre_conn, table='mytable', properties=properties)

but the table is not ordered in PostgreSQL by date. Is there a way to keep the dataframe order in the PostgreSQL table?

Ivan
  • 19,560
  • 31
  • 97
  • 141
  • 2
    Data in tables has no specific order. You only have a guaranteed order if you specify an `order by` clause when selecting. – Mark Rotteveel Nov 17 '17 at 19:53

0 Answers0