0

a really strange thing happend to me with Connector/python and I couldn't find any explanation on the Internet.

I finished and closed first part of my program - database analysis. I've spent a lot of time with profilling to decrease to required time and it worked. Then I started the second part of the program, after several day I had to execute the first part to get data processed. But it wen't very slow. I knew I haven't made any important changes to that part.

So I spent several hours going through git log and checkouting to previous version and found the last commit, with fast analysis.

Ouput of the diff:

-  insertq = "INSERT INTO `sp_domains` (domain) VALUES (%s) ON DUPLICATE KEY UPDATE domain=domain"     
+  insertq = """
+  INSERT 
+  INTO `sp_domains` (domain)
+     VALUES (%s) 
+     ON DUPLICATE KEY UPDATE domain=domain
+  """

This is the only change I've made in the shared class and it really is the speed difference reason. I just can't figure out, what happend by using tripple quotation notation. Is it something with executemany(...) method, which is use to execute the query?

Thank you for explanation

dakov
  • 1,039
  • 2
  • 12
  • 33

1 Answers1

1

I think it has to do with turning your one query with 4 inline parts into one query with multiple parts per line. The executemany(...) may have to do additional processing to strip whitespace, newlines and tabs to ensure that it compresses correctly to the original statement (more than just rearranging the string but additional vulnerabilities, idk). If you want to write it that way, do the string processing yourself before hand with split and join. Or,

From here: Use implicit continuation, it's more elegant:

def f():
    s = ('123'
         '456')
    return s

....you can see if this method is any faster.

Community
  • 1
  • 1
blakev
  • 4,154
  • 2
  • 32
  • 52