Disable Doctrine 2 lazy loading when using JMS Serializer?

Question

Im using Doctrine 2 ORM in my Zend project and need to serialize my Entities to JSON in several cases.

ATM i use the Querybuilder and join all tables i need. But my serializer causes doctrine to lazy load every associated Entity which results in pretty huge data amounts and provokes recursion.

Now im looking for a way to totally disable Doctrines lazy loading behavior.

My way to select data would be the following:

$qb= $this->_em->createQueryBuilder()
            ->from("\Project\Entity\Personappointment", 'pa')
            ->select('pa', 't', 'c', 'a', 'aps', 'apt', 'p')
            ->leftjoin('pa.table', 't')
            ->leftjoin('pa.company', 'c')
            ->leftjoin('pa.appointment', 'a')
            ->leftjoin('a.appointmentstatus', 'aps')
            ->leftjoin('a.appointmenttype', 'apt')
            ->leftjoin('a.person','p')

I would like my resultset to only contain the selected tables and associations.

Any help would be greatly appreciated.

If you use JMS Serializer, rely on my Answer. If you want to avoid JMS Serializer altogether, rely on Exanders Answer (especially the comments). — Christian Huber, Jul 31 '12 at 13:47

score 12 · Answer 1 · edited Apr 22 '17 at 08:58

In the latest version of JMSSerializer, the place you should look at is

JMS\Serializer\EventDispatcher\Subscriber\DoctrineProxySubscriber

instead of

Serializer\Handler\DoctrineProxyHandler

To override the default lazy load behavior, one should define his own event subscriber.

In your app/config.yml add this:

parameters:
    ...
    jms_serializer.doctrine_proxy_subscriber.class: Your\Bundle\Event\DoctrineProxySubscriber

you can copy the class from JMS\Serializer\EventDispatcher\Subscriber\DoctrineProxySubscriber to Your\Bundle\Event\DoctrineProxySubscriber and comment out the $object->__load(); line

public function onPreSerialize(PreSerializeEvent $event)
{
    $object = $event->getObject();
    $type = $event->getType();

    // If the set type name is not an actual class, but a faked type for which a custom handler exists, we do not
    // modify it with this subscriber. Also, we forgo autoloading here as an instance of this type is already created,
    // so it must be loaded if its a real class.
    $virtualType = ! class_exists($type['name'], false);

    if ($object instanceof PersistentCollection) {
        if ( ! $virtualType) {
            $event->setType('ArrayCollection');
        }

        return;
    }

    if ( ! $object instanceof Proxy && ! $object instanceof ORMProxy) {
        return;
    }

     //$object->__load(); Just comment this out

    if ( ! $virtualType) {
        $event->setType(get_parent_class($object));
    }
}

Thanks. This was helpful. However PersistentCollection objects are still being loaded. Any solution to disable this ? — nikora, Sep 16 '15 at 14:33
@nkobber, Hi I too am facing same problem, did you find solution to this ? — vishal, May 09 '16 at 07:18

score 9 · Accepted Answer · answered Jul 27 '12 at 07:01

After having looked for the answer in Doctrine, my team figured out that the JMS Serializer was the "problem". It triggered the use of Doctrine Proxies automatically. We wrote a Patch for JMS Serializer to avoid the Lazy Loading.

We implemented our own DoctrineProxyHandler which just doesn't trigger Doctrines lazyloading mechanism and registered it within our SerializationHandlers Array.

class DoctrineProxyHandler implements SerializationHandlerInterface {

public function serialize(VisitorInterface $visitor, $data, $type, &$handled)
{
    if (($data instanceof Proxy || $data instanceof ORMProxy) && (!$data->__isInitialized__ || get_class($data) === $type)) {
        $handled = true;

        if (!$data->__isInitialized__) {

            //don't trigger doctrine lazy loading
            //$data->__load();

            return null;
        }

        $navigator = $visitor->getNavigator();
        $navigator->detachObject($data);

        // pass the parent class not to load the metadata for the proxy class
        return $navigator->accept($data, get_parent_class($data), $visitor);
    }

    return null;
}

Now i can simply select my table, join the associations i need - and my JSON will contain just the data i selected instead of infinite depth associations and recursions :)

$qb= $this->_em->createQueryBuilder()
        ->from("\Project\Entity\Personappointment", 'pa')
        ->select('pa', 't', 'c', 'a')
        ->leftjoin('pa.table', 't')
        ->leftjoin('pa.company', 'c')
        ->leftjoin('pa.appointment', 'a')

JSON will just contain

{  
  Personappointment: { table {fields}, company {fields}, appointment {fields}}
  Personappointment: { table {fields}, company {fields}, appointment {fields}}
  Personappointment: { table {fields}, company {fields}, appointment {fields}}
  .
  .
}

Be careful, we figured out that there are several more places where lazy loading is triggered, although these are used less often. Can lead to strange behavior if ignored. — Christian Huber, Nov 02 '12 at 07:02
Specifically when collections of data are loaded (Doctrine\ORM\PersistentCollection). — Ben Waine, Nov 17 '12 at 19:05
Thanks for is answer, which I strongly inspired to build this component: https://github.com/alcalyn/serializer-doctrine-proxies which spots also another place where there is autoloading (see class DoctrineProxySubscriber) — Alcalyn, Jul 26 '16 at 13:05

score 4 · Answer 3 · answered Jul 23 '12 at 19:31

4

This may very well be called an ugly crutch, but you could just select() the data that you really need, then hydrate the result to an array using the getArrayResult() method of the Query object...

answered Jul 23 '12 at 19:31

Exander

852
1
7
17

We tried this method at first. But we have a pretty large DB Model and use Doctrines Mapped Superclass. So this simple Method works as long as we dont select multiple Entities which have equal named fields. For Example: We are using Doctrines "Version" Field, which exists in nearly every Entity. I would have to manually use "select as" entityname_version which is - as you said - an ugly crutch :( – Christian Huber Jul 25 '12 at 12:28
1

I'm not sure, but won't it actually return an array of arrays, where each second-level array represents a hydrated object?.. Which means there won't be any field name clashes if you select by tablename.* – Exander Jul 25 '12 at 14:06
Doesn't the data of associated entities get stored in a separate array, which is, in turn, stored under the approriate key in the array representing the object cobntaining the association (although I'm not too sure about bidirectional associations). – Exander Jul 25 '12 at 14:15
Oh, and I meant selecting "entityName1, entityName2 etc." , not "entityName1.*, entityName2.*". Where entityName2 may actually be entityName1.someAssociatedEntity1 – Exander Jul 25 '12 at 14:16
Although I don't actually remember if the resulting return value will differ based on whether you select entityName.* - or just entityName... – Exander Jul 25 '12 at 15:04
In fact, im using some kind of JMS Serializer, which is (and thats a pity) not capable of putting the "Entities name." in front of the field. "TCompany.Version" field is getting reduced to "Version" :-/ – Christian Huber Jul 26 '12 at 06:53
Do you actually need to use that serializer when you're working with arrays, not with entities? – Exander Jul 26 '12 at 10:11
thx to you too. Yes, i need the serializer cause i want JSON - and the PHP json_encode() method is useless since it wont work with private/protected doctrine properties. Anyway, i recommend everyone not to use Doctrine when there is JSON serialization needed. – Christian Huber Jul 27 '12 at 08:10
1

err...it seems I'm missing something. If you're hydrating the result to an array, where will the private properties come from when you're serializing the array to JSON?.. – Exander Jul 27 '12 at 08:31
You are right too. I tried it and the hydrated array just contains the tables i joined manually. So we could dispose of JMS Serializer altogether. How could i be so blind? – Christian Huber Jul 31 '12 at 13:46
the other drawback of hydrating to an array is if you're also using a serialization subscriber. So far, I've not found a way to configure one of those to hook into an array, only an object. – jrg Mar 07 '15 at 20:09

score 3 · Answer 4 · answered Jul 23 '12 at 09:23

3

When using Doctrine's query builder, you can't disable lazy loading of linked model classes. If you want to bypass such behavior, you better have to request data with Doctrine's DBAL.

Don't use \Doctrine\ORM\QueryBuilder but \Doctrine\DBAL\Query\QueryBuilder.

$qb = new QueryBuilder($this->_em->getConnection());
$expr = $qb->expr();

$qb->select('pa.*', 't.*', 'c.*', 'a.*', 'aps.*', 'apt.*', 'p.*')
   ->from('person_appointment', 'pa')
   ->leftJoin('pa', 'table', 't', $expr->eq('pa.table_id', 't.table_id'))
   // put other joints here
   // ...
   ->leftjoin('a', 'person', 'p', $expr->eq('a.person_id', 'p.person_id'));

answered Jul 23 '12 at 09:23

Florent

12,310
10
49
58

Hi! Thanks, this one looked good at first - but it seems that the DBAL qb is just capable of building SQL statements that i could execute with PDO afterwards.. All ORM functionality is missing. – Christian Huber Jul 25 '12 at 12:22
The problem is that what you want is to ignore ORM functionalities! You can't load a model class with Doctrine ORM without loading related model classes. – Florent Jul 25 '12 at 13:57
1

Maybe im approaching the problem from the wrong side, and should start modifying the serializer. But the Serializer cant know if the association it calls is allready there or lazy loaded when called. My wish would be that an association that was not mentioned in a JOIN is just NULL instead of being lazy loaded.. – Christian Huber Jul 26 '12 at 06:58
I awarded your answer with the bounty since you made one point clear: No Doctrine ORM without Lazy Loading. thx – Christian Huber Jul 27 '12 at 08:05
Thanks. I think you don't approach your problem the right way. Class loading isn't the issue here. The performance drop is pretty small: if you load 10 classes more it's nothing compared to the 50 (maybe more) loaded by Doctrine or even your application framework. – Florent Jul 27 '12 at 12:28
chuber50: "No Doctrine ORM without Lazy Loading" - not true. doctrine supports both lazy loading and eager loading, by default lazy loading – safrazik Jun 20 '13 at 10:29

score 1 · Answer 5 · answered Oct 10 '16 at 19:44

Case you want pragmatically use your or default subscriber,

@DavidLin answer:

you can copy the class from JMS\Serializer\EventDispatcher\Subscriber\DoctrineProxySubscriber to Your\Bundle\Event\DoctrineProxySubscriber and comment out the $object->__load(); line

<?php

/*
 * Copyright 2013 Johannes M. Schmitt <schmittjoh@gmail.com>
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

namespace Your\Bundle\Event;

use Doctrine\ORM\PersistentCollection;
use Doctrine\ODM\MongoDB\PersistentCollection as MongoDBPersistentCollection;
use Doctrine\ODM\PHPCR\PersistentCollection as PHPCRPersistentCollection;
use Doctrine\Common\Persistence\Proxy;
use Doctrine\ORM\Proxy\Proxy as ORMProxy;
use JMS\Serializer\EventDispatcher\PreSerializeEvent;
use JMS\Serializer\EventDispatcher\EventSubscriberInterface;

class AvoidDoctrineProxySubscriber implements EventSubscriberInterface
{
    public function onPreSerialize(PreSerializeEvent $event)
    {
        $object = $event->getObject();
        $type = $event->getType();

        // If the set type name is not an actual class, but a faked type for which a custom handler exists, we do not
        // modify it with this subscriber. Also, we forgo autoloading here as an instance of this type is already created,
        // so it must be loaded if its a real class.
        $virtualType = ! class_exists($type['name'], false);

        if ($object instanceof PersistentCollection
            || $object instanceof MongoDBPersistentCollection
            || $object instanceof PHPCRPersistentCollection
        ) {
            if ( ! $virtualType) {
                $event->setType('ArrayCollection');
            }

            return;
        }

        if ( ! $object instanceof Proxy && ! $object instanceof ORMProxy) {
            return;
        }


        //Avoiding doctrine lazy load proxyes
        //$object->__load();

        if ( ! $virtualType) {
            $event->setType(get_parent_class($object));
        }
    }

    public static function getSubscribedEvents()
    {
        return array(
            array('event' => 'serializer.pre_serialize', 'method' => 'onPreSerialize'),
        );
    }
}

And initialize the serialize like this:

$serializer = JMS\Serializer\SerializerBuilder::create()
    //remove this to use lazy loading 
    ->configureListeners(function(JMS\Serializer\EventDispatcher\EventDispatcher $dispatcher) {
        $dispatcher->addSubscriber(new Your\Bundle\Event\AvoidDoctrineProxySubscriber());
    })  
    // !remove this to use lazy loading 
    ->build();

//and serialize the data with/without Lazy

serializer->serialize($data, 'json');

Disable Doctrine 2 lazy loading when using JMS Serializer?

5 Answers5

Linked