EPFImporter creates table but imports nothing

Question

I'm using Apple's EPFImporter tool http://www.apple.com/itunes/affiliates/resources/documentation/epfimporter.html

It's a Python script that will take space separated EPF file lists and import them into my database.

Here's what I have:

Braden-Keiths-MacBook-Pro:~ bradenkeith$ ./EPFImporter.py /Users/bradenkeith/Downloads/popularity20120314

Here's what the CLI returns:

2012-03-14 22:12:28,748 [INFO]: Beginning import for the following directories:
    /Users/bradenkeith/Downloads/popularity20120314 
2012-03-14 22:12:28,748 [INFO]: Importing files in /Users/bradenkeith/Downloads/popularity20120314 
2012-03-14 22:12:28,749 [INFO]: Starting import of /Users/bradenkeith/Downloads/popularity20120314... 
2012-03-14 22:12:28,749 [INFO]: Beginning full ingest of epf_application_popularity_per_genre (2000491 records) 
2012-03-14 22:14:28,774 [INFO]: ...at record 1797000... 
2012-03-14 22:16:02,152 [INFO]: Full ingest of epf_application_popularity_per_genre took 0:03:33.402408 
2012-03-14 22:16:02,196 [INFO]: Import of popularity20120314 completed at: 12-03-14 22:16:02 
2012-03-14 22:16:02,196 [INFO]: Total import time for popularity20120314: 0:03:33.44 
2012-03-14 22:16:02,196 [INFO]: Total import time for all directories: 0:03:33.44

The tool was capable of creating the database. It just won't add any of the entries to the database. It obviously sees 2mil+ records, and spends the time combing through them... deleting and merging the blank tables... but it's just that - tables are still blank. I thought possibly it's a permissions thing with mySQL. I double checked and made sure everything was granted to the user account I was using. Still nothing.

Any ideas of what this might could be?

Did you resolve this? I'm finding the same problem but changing autocommit doesn't seem to work for me, as suggested by maxperreault. — Tonester, Apr 01 '12 at 21:47
What seems to be working for me is to delete whatever is in Users/username/.python-eggs/ per http://stackoverflow.com/questions/3061277/python-mysql-wrong-architecture-error What I'm thinking is there were different bits of mysql installed or something... some library was in the wrong bit mode. But it worked for me after deleting what was cached in there. — bradenkeith, Apr 03 '12 at 01:34
Thanks for getting back to me. That didn't work for me unfortunately, but maybe a fresh install is the next step. — Tonester, Apr 03 '12 at 13:24

score 11 · Answer 1 · answered Mar 07 '13 at 16:44

You can get it to work by altering EPFIngester.py according to:

Find function

def _populateTable(self, tableName, resumeNum=0,
isIncremental=False, skipKeyViolators=False):

In the function, within the while loop, find row:
```
cur = conn.cursor()
```
Under it insert:
```
cur.connection.autocommit(True)
```

The altered source should look like:

...
cur = conn.cursor()
cur.connection.autocommit(True)
colVals = unicode(", ".join(stringList), 'utf-8')
....

score 6 · Accepted Answer · answered Mar 23 '12 at 18:24

6

The EPFImporter was made in 2010. At the time, the latest version of MySQLdb set autocommit to true. The version of MySQLdb you are using is most likely a newer version where autocommit is set to false.

answered Mar 23 '12 at 18:24

maxperreault

96
4

This was exactly the problem in my case. The solution is Either you set the global autocommit flag to true as mentioned here , Else follow the solution given in the below answer by @Chris Nilsson – sandy.sk Nov 26 '14 at 11:16

score 0 · Answer 3 · answered Jul 10 '13 at 08:15

0

What storage engine are you using? Had exactly the same problem. Switching the storage engine from InnoDB to MyISAM solved it.

answered Jul 10 '13 at 08:15

Daniel Powers

111
1
3

EPFImporter creates table but imports nothing

3 Answers3