Stack Overflow
Stack Overflow

Questions tagged [executemany]

A method to prepare a database query or command and execute it against all parameters found in a sequence or mapping of parameters.

135 questions
18
votes
4 answers

how to transform pandas dataframe for insertion via executemany() statement?

I have a fairly big pandas dataframe - 50 or so headers and a few hundred thousand rows of data - and I'm looking to transfer this data to a database using the ceODBC module. Previously I was using pyodbc and using a simple execute statement in a…
Colin O'Brien
  • 2,175
  • 5
  • 20
  • 26
17
votes
1 answer

For Loop or executemany - Python and SQLite3

I have started to learn Python and SQL recently and have a question. Using Python with SQLite3 I have written the following code: # Use sqlite3 in the file import sqlite3 # Create people.db if it doesn't exist or connect to it if it does exist with…
Simon Roadknight
  • 173
  • 1
  • 1
  • 4
15
votes
1 answer

is it possible to fetch all the lastrowids after executemany insert in mysqldb?

I cannot be sure whether the ids generated is continously,if not,is there any otherway to get them? class BaseDao(object): def __init__(self,pooldb): self.pooldb = pooldb def insertmany(self,sql,args): conn,cur = None,None …
bigwesthorse
  • 225
  • 2
  • 8
9
votes
2 answers

SQLite3 Python: executemany SELECT

I'm trying to get all the rows out of a table in one line with some WHERE constraints using the executemany function import sqlite3 con = sqlite3.connect('test.db') cur = con.cursor() cur.execute('CREATE TABLE IF NOT EXISTS Genre (id INTEGER…
Brandon Nadeau
  • 3,568
  • 13
  • 42
  • 65
8
votes
1 answer

pyodbc INSERT INTO from a list

I am trying to insert data into an Access mdb file using a list as the source for the values. cursor.execute("select * from Components") cursor.executemany(""" INSERT INTO Components ([Database…
user3579106
  • 251
  • 1
  • 2
  • 9
7
votes
2 answers

Python + MySQLdb executemany

I'm using Python and its MySQLdb module to import some measurement data into a Mysql database. The amount of data that we have is quite high (currently about ~250 MB of csv files and plenty of more to come). Currently I use cursor.execute(...) to…
lhahne
  • 5,909
  • 9
  • 33
  • 40
5
votes
1 answer

Any other way to import data files(like .csv) in python sqlite3 module ? [not insert one by one]

In sqlite3's client CLI, there is " .import file TABLE_name " to do it. But, I do not want to install sqlite3 to my server at present. In python sqlite3 module, we can creat and edit a DB. But, I have not found a way to import data-file to a TABLE,…
user343638
  • 51
  • 1
  • 2
5
votes
2 answers

MySqlDb throws Operand should contain 1 column(s) on insert ignore statement

While looking at some of the websocket methods that stack exchange offers, I wanted to save a few data points into a MySQL database. However, when I attempt to run an executemany command, I get the following…
Andy
  • 49,085
  • 60
  • 166
  • 233
5
votes
2 answers

executemany for MySQLdb error for large number of rows

I'm currently running a script to insert values (a list of tuples) into a MySQL database, using the execute many function. When I use a small number of rows (`1000), the script runs fine. When I use around 40,000 rows, I receive the following…
Matt
  • 1,194
  • 1
  • 18
  • 18
4
votes
2 answers

(fast_executemany = True) Error "[ODBC Driver 17 for SQL Server]Invalid character value for cast specification (0) (SQLExecute)')"

I'm using (executemany) function from pyodbc to populate data into an mssql database. This is my code: def populate_database(conn): tuples = [ ('2020-04-13 00:50:42', 'AirShoppingRQ', 'ALEY', '2020-05-23', '', '', 'BRU-BLQ', ''), …
Daniel Pages Chacon
  • 61
  • 1
  • 6
4
votes
2 answers

How to handle exception with executemany (MySQL and Python)

I have a python script using executemany to bulk insert rows into a MySQL table. The data is retrieved from different APIs, so every now and then there is unexpected data which leads to a row causing exception. If I understand correctly - when…
Aamit
  • 181
  • 4
  • 16
4
votes
2 answers

Python MySQL DB executemany does not work on one value

I am trying to do a batch insert for a single value (row), I am trying to use the executemany function to do so, but it will not work it returns TypeError: not all arguments converted during string formatting. However, when I add an extra value it…
jped
  • 486
  • 6
  • 19
4
votes
1 answer

Python cx_Oracle. banging head with executemany()

I spent most of yesterday looking at questions here and around the web and Couldn't really, still, figure out what I'm missing. I'm pretty sure it has to be something really stupid but I'm burn out by now. So, code: temp_table_name =…
jdq2013
  • 43
  • 1
  • 3
4
votes
2 answers

psycopg2 executemany with simple list?

I'm trying to use psycopg2 executemany for a simple multi-insert but I can only make it work using dict and not "plain" sequence of values: # given: values = [1, 2, 3] ; cursor = conn.cursor() # this raises TypeError: 'int' object does not support…
gst
  • 431
  • 1
  • 4
  • 16
4
votes
2 answers

Sqlite executemany and DELETE

Execute many seems to be very slow with deletion (Insertion is fine) and I was wondering if anyone knows why it takes so long. Consider the code below: import sqlite3 db = sqlite3.connect("mydb") c = db.cursor() c.execute("DROP TABLE IF EXISTS…
Joran Beasley
  • 110,522
  • 12
  • 160
  • 179
1
2 3
…
8 9