1

enter image description hereIn my Symfony2 project, using Doctrine2, I got a User class extending the FOSUserBundle's User class, which has a lot of associations, as an adress, images,commands, etc..

class User extends BaseUser{

/**
 * @ORM\Id
 * @ORM\Column(type="integer")
 * @ORM\GeneratedValue(strategy="AUTO")
 * @Expose
 */
 protected $id;


/**
 *
 * @ORM\OneToMany(targetEntity="ANG\CommandBundle\Entity\Command", cascade={"persist", "remove"}, mappedBy="customer")
 *@Expose
 *
 */
private $commands;


/**
 *
 * @ORM\OneToOne(targetEntity="ANG\FileBundle\Entity\Image",  cascade={"remove"})
 * @ORM\JoinColumn(onDelete="SET NULL")
 * @Assert\Valid()
 * @MaxDepth(1)
 * @Expose
 */
private $banner;


/**
 *
 * @ORM\OneToOne(targetEntity="ANG\FileBundle\Entity\Image",  cascade={"remove"})
 * @ORM\JoinColumn(onDelete="SET NULL")
 * @Assert\Valid()
 * @MaxDepth(2)
 * @Expose
 */
private $avatar;


/**
 *
 * @ORM\OneToMany(targetEntity="ANG\MainBundle\Entity\Address", cascade={"persist", "remove"}, mappedBy="customer")
 *@MaxDepth(1)
 *@Expose
 */
private $address;


/**
 *
 * @ORM\OneToMany(targetEntity="ANG\MainBundle\Entity\Invitation" , mappedBy="customerSender", cascade={"persist", "remove"})
 *
 */
protected $invitationsSend;

...

I realised that when I do $this->getUser() in a controller, the doctrine lazy loading executes 128 queries... (which is far too much to be acceptable)

So I created my own function in the UserRepository, using setHint(Query::HINT_FORCE_PARTIAL_LOAD, true) to avoid the lazy loading.

    public function testGetUser($id){

    $em=$this->getEntityManager();
    $qb=$em->createQueryBuilder()
        ->select('cs')
        ->from('ANGCustomerBundle:User', 'cs')
        ->where('cs.id = :id')
        ->setParameter('id', $id);


    return $qb->getQuery()->setHint(Query::HINT_FORCE_PARTIAL_LOAD, true)->getOneOrNullResult();
}

My problem : That function does not work for the logged in user.

Example:

I'm logged in with the User having ID = 3

Working on all users except me

For every ID != 3, the abose request is working.

$user = $em->getRepository('ANGCustomerBundle:User')->testGetUser(4); returns: {id: 4, username: "coy", email: "n****@gmail.com"}

Result:

  • 2 queries
  • User object without any associations

Not working on the logged in user...

But the same request on the logged in user $user = $em->getRepository('ANGCustomerBundle:User')->testGetUser(3); returns

{"id":3,"username":"bolz","email":"h******@gmail.com","images":[{"id":15,"url":"png"},{"id":20,"url":"jpeg"}],"commands":[{"id":1,"date_creation":"2016-03-25T15:52:40+0100", .....

Result:

  • 128 queries
  • The entire User object, with all his associations. We can see that we have the user's command, but we also have all the command's associate entities and so on ...

It looks like the HINT_FORCE_PARTIAL_LOAD doesn't apply on the logged in user .

Just to conclude, this problem is also present if I request on all the users together.

public function findALLs(){
    $em=$this->getEntityManager()
        ->createQueryBuilder()
        ->select('cs')
        ->from('ANGCustomerBundle:User', 'cs')
        ->getQuery()->setHint(Query::HINT_FORCE_PARTIAL_LOAD, true)->getResult();
}

For all users except the logged on : username, email, id

But like for the 1st function, we have the entire structure for the logged in user, doing once again, 128queries...

I've been stuck for a week now on this and really don't understand what's happening here.

I really thank you for reading this long post, and hope someone could help me on this one.

With much thanks, Bastien.

Bastien H.
  • 13
  • 5
  • could you test with `$em->getRepository('ANGCustomerBundle:User')->find(4)` – Mohamed Ramrami Mar 31 '16 at 15:01
  • it returns me the entire User object, as the 'find' function loads every associations. But I only want the 'basic' User object, wihtout the associate entities. – Bastien H. Mar 31 '16 at 15:41
  • To be more specific, here I get User.username, User.email, User.id, (what I want) but also user.command.*, user.avatar.*, user.adress.*, user.command.product.*, user.command.product.image.* etc... really all the associate entities, that's what I want to avoid – Bastien H. Mar 31 '16 at 15:48
  • @Expose is part of JMSBundle, its to select attributes which are send or not to frontend, as my application is a REST API. [documentation here](http://jmsyst.com/libs/serializer/master/reference/annotations) But I don't think that it's a cause here, as it act after the request was done, and then filter the result – Bastien H. Mar 31 '16 at 15:55
  • where/how do you test the returned result ? – Mohamed Ramrami Mar 31 '16 at 17:12
  • I use Symfony, so I got the Symfony's profiler which shows me every querie made. And I can see the request's result in my browser. – Bastien H. Apr 01 '16 at 07:41
  • But I think that there's some kind of service or something in FOSUserBundle which kind of force the full load of logged in user. I don't really understand what's going on here – Bastien H. Apr 01 '16 at 07:53
  • So you have some functional test, or you hit the route from a browser or what? Sorry if I ask a lot of questions, just trying to help. – Mohamed Ramrami Apr 01 '16 at 08:39
  • No problem, thank you for trying to help ! Yes, I hit the route from a browser, then in the request's response, I can see the object returned, and there's also a link like '/app_dev.php/_profiler/d5436e' which allows me to access the Symfony Profiler, where there is a full recap of what happened in Symfony for this request. (time, events, queries,routing, etc...) – Bastien H. Apr 01 '16 at 08:49
  • ok try to do this: after you call find , do a `dump($user); die();` and see if the user object contains all asociations or not. – Mohamed Ramrami Apr 01 '16 at 09:04
  • I made a screenshot of the response : [here[http://i.stack.imgur.com/QTcUL.png] I think that at the moment, all the associations are not load. But I don't really understand how and why. As ->find() should load the entire object with his associations ? I must have missed something on lazy loading. – Bastien H. Apr 01 '16 at 10:07
  • So the associations are loaded or not ? I couldnt see it from the screenshot. The lazy loading means the associations will not be loaded untill you access them. – Mohamed Ramrami Apr 01 '16 at 10:17
  • I think they are not loaded, as commands, images, etc.. are empty. There's not command.product etc... So you must be right ! I did a $this->getUser() and it dump the same thing. If at the end of my controller I return $user, then 128queries are executed, but if I return 'OK', only 1 query is executed. So the problem may be, as u supposed, with JMS, or something with the return, so the response's sending – Bastien H. Apr 01 '16 at 10:20
  • And this is what you want right... So we're getting closer to the probleme, now remove `dump($user); die();` and remove `@Exposed` from associations fields like command ... and see what hapens – Mohamed Ramrami Apr 01 '16 at 10:23
  • Yes it's what I want ^^ Ok I removed all the @Expose but the result is the same, if I don't return the $user => only 1 query. But if I return $user, then 130queries are done (on every associate entity). But I'm not completly sure that JMSSerializer is really turned off, so I'm gonna make sure of this. The 'return' action could trigger an action in doctrine ? If not, it's JMS for sure . – Bastien H. Apr 01 '16 at 10:34
  • Are you using FOSRestBundle also ? – Mohamed Ramrami Apr 01 '16 at 10:38
  • Yes I do! Btw, thank you again for your time and your reactivity =) – Bastien H. Apr 01 '16 at 10:39
  • ok, no problem I'm glad to help :) – Mohamed Ramrami Apr 01 '16 at 10:40
  • pliz update your question with FOSRestBundle config – Mohamed Ramrami Apr 01 '16 at 10:41

1 Answers1

0

The problem is with JMSSerializerBundle, which use \JMS\Serializer\EventDispatcher\Subscriber\DoctrineProxySubscriber in which there is a call to $object->__load(); that trigger lazy loading.

See this question where people had the same problem and managed to make it work.

Hope it helped.

Community
  • 1
  • 1
Mohamed Ramrami
  • 12,026
  • 4
  • 33
  • 49
  • I was not able at the moment to overwrite the ProxySuscriber with mine, it doesn't work I don't know why, so I directly commented the `$object->__load();` line. And now I get 31queries (130 without the comment) and the User object seems to load associated entities for '2 levels of deepness'. I mean, I have user.commands , user.commands[i].producer, but in producer I only have the id, whereas without commenting the line, I can also access user.commands[i].producer.avatar for example, or even user.commands[i].producer.products[j].categorie etc... So it seems that this is 50% working ^^ – Bastien H. Apr 01 '16 at 12:37
  • I think I start to understand a bit better ^^ It looks like I get all my OneToOne, OneToMany and ManyToMany relations but not the ManyToOne. So for example in my response object I get user.commands (OneToMany) , user.commands.command_packagings (OneToMany again) , but I don't have user.commands.producer (just the id) because it's a ManyToOne. I'm not sure of why it's working like this, but it seems to be kind of a logic behaviour. I'll try to see if I can avoid getting those 1to1 1to* & \*to\* relations – Bastien H. Apr 01 '16 at 13:07
  • But there's still something I don't understand. If I run my request 'findAlls' (the last one I linked in my question),still with the modificated ProxySucriber, I got 32queries : 1 for selecting all the users (without any associations) and 31for the logged user. So I got id,username,email for all users, but for the logged in user, there's still that kind of '2 level deep object' I discribed in my last comment. I don't know why there are different behaviours between the actual logged user and the others. That's why I thought at the beggining that it came from FOSUserBundle – Bastien H. Apr 01 '16 at 13:32
  • Maybe the other users have no 'commands' in database that's why – Mohamed Ramrami Apr 01 '16 at 14:00
  • yes they all have commands. I take commands for example but it works for every fields, commands, groups, friends, etc.. In fact when I dump() the request's response, every users looks the same. But it looks like for some reason, before sending the response (before the serializing maybe) more queries are executed on the logged in user – Bastien H. Apr 01 '16 at 14:14
  • [1]: http://i.stack.imgur.com/rKTes.png here you can see the request's response, the user being logged is 'bolz', 'id:3'. You can see the difference between this user and the 2 others – Bastien H. Apr 01 '16 at 14:21
  • which version of FOSUserBundle are you using. I can' t find BaseUser in their github repo. – Mohamed Ramrami Apr 01 '16 at 14:35
  • 2.0, here is the doc I followed : [doc](http://symfony.com/doc/current/bundles/FOSUserBundle/index.html) – Bastien H. Apr 01 '16 at 14:39
  • ok i found it, it's just `use FOS\UserBundle\Entity\User as BaseUser;` – Mohamed Ramrami Apr 01 '16 at 14:39
  • Did you override the method `serialize`or something in your user class? – Mohamed Ramrami Apr 01 '16 at 14:40
  • No. I just added `FOS\UserBundle\Model\User: exclusion_policy: ALL properties: id: expose: true username: expose: true email: expose: true` Which allows me to only send the id/username/email for User and not the rest. But I put it away, and the behaviour is the same, exceot that I got password, usernameCanonical etc.. for the users, but still no associations for all users, except the logged one. So exactly same behaviour, I don't think that it changes something. And I didn't do anything else concerning the serialization – Bastien H. Apr 01 '16 at 14:50
  • Did you configure JMSSerializer to override third party MetaData ? – Mohamed Ramrami Apr 01 '16 at 15:35
  • ok try this for debugging reason : after your `find` loop over the users list and do a `dump(get_class($user));` and `die();` after the end of loop. – Mohamed Ramrami Apr 01 '16 at 15:47
  • try also this one in your controller `dump(get_class($this->getUser())); die();` – Mohamed Ramrami Apr 01 '16 at 15:53
  • "ANG\CustomerBundle\Entity\User" – Bastien H. Apr 01 '16 at 15:59
  • The behavior for the logged user is correct because you get what you exposed with `@Expose`, still to understand what happens for the other users. – Mohamed Ramrami Apr 01 '16 at 16:43
  • nop, the correct behaviour is for the other users. As in my request there's `setHint(Query::HINT_FORCE_PARTIAL_LOAD, true)`, so no associations should be returned, not even the command.id for example. This is what I expect, no related entities except the one I explicitly select with a join in my request – Bastien H. Apr 04 '16 at 09:51
  • the `find`is returning the expected result with no associations! you can see that if you do a dump after the find. But before sending the response back, the serialization is triggered and because of the `@Expose` the lazy loading is triggered also and you fetch the associations. You can test this by removing `@Expose` and clear the cache before you test. – Mohamed Ramrami Apr 04 '16 at 10:42
  • It looks like there is no difference, except that now we have the password, emailCanonical, userCanonical, etc... But the structure is the same, no associated entities except for the logged in user. Here you can see a screen of the dump of $users, with $user=repo->findAlls() , the logged in user is 'bolz' [1]: http://i.stack.imgur.com/Mg3Mu.png – Bastien H. Apr 04 '16 at 11:28
  • do you have some annotation `@ExclusionPolicy` above your User class ? – Mohamed Ramrami Apr 04 '16 at 11:36
  • I had one, but I removed it when I removed the `@Expose` cause if there is `@ExclusionPolicy` and no `@Expose`, then nothing will be returned. – Bastien H. Apr 04 '16 at 11:43
  • Try to put it back, and remove `@Expose`only from associations fields like commands ... – Mohamed Ramrami Apr 04 '16 at 11:44
  • I tried, but it's not the behaviour I want. In fact, with no `@Expose` on commands, I don't have user.commands in the response, but it's still there in the dump,and it's still a PersistentCollection for the logged in user. I won't be able to get the commands when I want them. The thing is I would like to get just the '1 level user' if I do a simple `createQueryBuilder() ->select('cs') ->from('ANGCustomerBundle:User', 'cs');` and I want for example user + his commands if I do the same request + `->leftJoin('cs.commands','cmd') ->addSelect('cmd')` – Bastien H. Apr 04 '16 at 11:56
  • I think I found the solution in the question you linked me, in the last answer, on the OnlyLoadedAssociationsExclusionStrategy. I used this class, and at the moment it is looks like it works. I continue my tests, but if it's ok I will mark the question as solved. Thanks again – Bastien H. Apr 05 '16 at 08:24