0

New daily csv file is constantly generated and written into a directory. The task is to automatically import that new csv everyday into the postgres table (so the PG table will be appended with one new record everyday). I have a python script that is good for importing ALL csv files. How can I just import the most recent csv?

import glob
import psycopg2

file_names = glob.glob('path/to/directory/*.csv')

con = psycopg2.connect(database="XXXX", user="XXXX", password="XXXX", host="XXXX")

for file_name in file_names:
    with open(file_name, 'r') as file_in:
        next(file_in)
        with con.cursor() as cur:
            cur.copy_from(file_in, "tbl_name", columns=('objectid', 'starttime', 'endtime', 'comments'), sep=",")
        con.commit()

con.close()
geopark
  • 25
  • 4

1 Answers1

1

I believe your problem is purely to find the latest file in a folder. If so, there is here another thread solving exactly that question

Ftisiot
  • 1,808
  • 1
  • 7
  • 13