6

I have a CSV file without headers and am trying to create a SQL table from certain columns in the file. I tried the solutions given here: Importing a CSV file into a sqlite3 database table using Python, but keep getting the error that col1 is not defined. I then tried inserting headers in my CSV file and am still getting a KeyError.

Any help is appreciated! (I am not very familiar with SQL at all)

Community
  • 1
  • 1
user2483176
  • 317
  • 1
  • 10
  • 21

2 Answers2

2

If the .csv file has no headers, you don't want to use DictReader; DictReader assumes line 1 is a set of headers and uses them as keys for every subsequent line. This is probably why you're getting KeyErrors.

A modified version of the example from that link:

import csv, sqlite3

con = sqlite3.connect(":memory:")
cur = con.cursor()
cur.execute("CREATE TABLE t (col1, col2);")

with open('data.csv','rb') as fin:
    dr = csv.reader(fin)
    dicts = ({'col1': line[0], 'col2': line[1]} for line in dr)
    to_db = ((i['col1'], i['col2']) for i in dicts)

cur.executemany("INSERT INTO t (col1, col2) VALUES (?, ?);", to_db)
con.commit()
Free Monica Cellio
  • 2,270
  • 1
  • 16
  • 15
0

This below code, will read all the csv files from the path and load all the data into table present in sqllite 3 database. 

   import sqllite3
   import io
   import os.path
   import glob
        cnx = sqlite3.connect(user='user', host='localhost', password='password',
                                        database='dbname')
        cursor=cnx.cursor(buffered= True);


        path ='path/*/csv' 
        for files in glob.glob(path + "/*.csv"):
            add_csv_file="""LOAD DATA LOCAL INFILE '%s' INTO TABLE tabkename FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n'  IGNORE 1 LINES;;;""" %(files)

            print ("add_csv_file: %s" % files)
            cursor.execute(add_csv_file)

        cnx.commit()
        cursor.close();
        cnx.close();

Let me know if this works.

user15051990
  • 1,835
  • 2
  • 28
  • 42