1

I am trying to optimize an import operation that uses Symfony 2 forms to validate data during the import process, a bit like this much simplified example:

/// For each imported row
foreach ($importedData as $row) {

    $user = new User;
    $userType = new UserType;

    // !! The 250 KB of memory allotted in this line is never released again !!
    $form = $symfonyFormFactory->create($userType, $user);

    // This line allots a bunch of memory as well,
    // but it's released when the EM is flushed at the end
    $form->bind($row);

    if (!$form->isValid()) {
        // Error handling
    }

    $entityManager->persist($user);
    $entityManager->flush();
}

Each one of those loops, memory usage is increasing by around 250 KB, which is crippling for large imports.

I have discovered that the memory leak is coming from the $form = $symfonyFormFactory->create(new UserType, $user); line.

EDIT: The bulk of the memory was being used by the entity manager, not the form component (see accepted answer). But the loop is still taking 55 KB each loop, which is better than 250 KB but could be better. Just not today.

mattalxndr
  • 9,143
  • 8
  • 56
  • 87

2 Answers2

2

Try to disable SQL logging to reduce memory usage

$entityManager->getConnection()->getConfiguration()->setSQLLogger(null)

Also i just found similar question here

Community
  • 1
  • 1
Vadim Ashikhman
  • 9,851
  • 1
  • 35
  • 39
0

Are you sure you don't want to release entity object? First of all, don't flush data in every iteration. Lets say persist 50 objects - I don't know how big is your import data - and flush them together, then clear the object by simply calling $entityManager->clear();

karlisba
  • 3
  • 3
  • I could be wrong but the `$entityManager->flush();` seems to free the entity memory. I'll check it again to make sure... – mattalxndr Oct 26 '12 at 21:11
  • The import should be able to be very, very long, so I need to flush on every row to save memory. – mattalxndr Oct 26 '12 at 21:12
  • Try $entityManager->getConnection()->getConfiguration()->setSQLLogger(null); Also, how many rows does the imported data contain? – Vadim Ashikhman Oct 26 '12 at 21:13
  • Then try to clear entity after X iterations because it takes some time too, if this doesn't work, than for a quick thought nothing else comes to mind, but at the moment I don't have an access to that code what I wrote. If this problem will be active on mondey I might help more. :) – karlisba Oct 26 '12 at 21:15
  • Yes, Vadim Ashikhman, might be the answer, but I wasn't able to remember that line. :) – karlisba Oct 26 '12 at 21:17
  • This line of code helped me with the persistance of 1500 entities, but i didnt use symfony form validation. However, there is still a problem with the release of memory of form objects. – Vadim Ashikhman Oct 26 '12 at 21:21
  • @VadimAshikhman The `$entityManager->getConnection()->getConfiguration()->setSQLLogger(null);` changed each loop's memory usage from 250 KB to around 75 KB! Please create an Answer so I can mark it as such. – mattalxndr Oct 26 '12 at 21:25