4

How can I create a loop with pandas read_csv?

I need to create a data loop to list and save to the database.

How can I do this loop with the data from a csv?

thank you all for your attention

produtos = pd.read_csv('tabela.csv', delimiter=';')
        for produto in produtos:
            print(produto['NOME'])
marcelo.delta
  • 2,730
  • 5
  • 36
  • 71
  • Can you clarify your question and add desired outcome? Or perhaps pseudocode – Biarys Feb 19 '19 at 02:57
  • I have a CSV with data, I need to save this data in a database. There are several rows with data and headed by names of possible columns. – marcelo.delta Feb 19 '19 at 03:03

3 Answers3

7

To iterate in the DataFrame resulted by calling pandas read_csv you should use the command iterrows() for iteration, as in the below example:

for produto in produtos.iterrows():
    print(produto['NOME'])
Fernandoms
  • 444
  • 3
  • 13
  • this doesnt iterate through pandas.read_csv. It iterates through pandas.DataFrame. read_csv simply reads the files and puts it into a DataFrame – Biarys Feb 19 '19 at 02:59
4

If you have files that you need to save, I recommend this

import os
import pandas as pd
from sqlalchemy import create_engine

engine = create_engine('sqlite://', echo=False)
path = "C:/path/to/directory"

# list all files in the directory, assuming this directory 
# contains only files csv files that you need to save 
for file in os.listdir(path):
    df = pd.read_csv(path+file)
    # some other data cleaning/manipulation

    # write dataframe to database
    df.to_sql("table_name", con=engine)

Alternative, you can create a list with all files locations and iterate through that one instead. More info on to_sql() and check out this answer

Biarys
  • 1,065
  • 1
  • 10
  • 22
0

If you can create loop with Pandas by column name:

produtos = pd.read_csv('tabela.csv', delimiter=';')
for i, produto in produtos.iterrows():
    print(produto['NOME'])

But if you want to insert directly on your database use sqlalchemy and function to_sql like this:

from sqlalchemy import create_engine
import pandas as pd
...
engine = create_engine("mysql://user:pwd@localhost/database")
produtos = pd.read_csv('tabela.csv', delimiter=';')
if_exists_do = 'append'
produtos.to_sql('table_name', con=engine, if_exists=if_exists_do)

Then it will be inserted on database. The var 'if_exists_do' can receive value 'replace' if you want this.

Anderson B. Magalhães
  • 204
  • 3
  • 8