4

I have two tables magazines with a field product_code, and another table issues. They have belongsToMany relationship.

Magazine model:

 public function issues()
 {
     return $this->hasMany('App\Issue');
 }

Issue model:

public function magazine()
{
    return $this->belongsTo('App\Magazine');
}

Currently I have a query where I get collections of issues grouped by magazine id and ordered by the date of the last issue.

$issues = Issue::orderBy('date', 'desc')->get()->groupBy('magazine_id');

This is how the result of my query looks like:

Collection {#431 ▼
  #items: array:23 [▼
    103 => Collection {#206 ▼
      #items: array:52 [▶]
    }
    106 => Collection {#216 ▶}
    124 => Collection {#452 ▶}
    112 => Collection {#451 ▶}
    115 => Collection {#450 ▶}
    123 => Collection {#449 ▶}
    107 => Collection {#448 ▶}
    113 => Collection {#447 ▶}
    117 => Collection {#446 ▶}
    109 => Collection {#445 ▶}
    110 => Collection {#444 ▶}
    121 => Collection {#443 ▶}
    120 => Collection {#442 ▶}
    114 => Collection {#441 ▶}
    116 => Collection {#440 ▶}
    118 => Collection {#439 ▶}
    126 => Collection {#438 ▶}
    125 => Collection {#437 ▶}
    119 => Collection {#436 ▶}
    122 => Collection {#435 ▶}
    105 => Collection {#434 ▶}
    111 => Collection {#433 ▶}
    104 => Collection {#432 ▶}
  ]
}

So, since I have 24 magazines, there are 24 collections of issues in the array, and each collection of issues belongs to one magazine. The collections are sorted by the date of the latests issue of each collection and issues inside of each collection are ordered by date as well. So, first collection in the array will be the one which has the latest issue in the table issues, the second collection will be the one which has the second latest issue in the same table and so on.

Since I will get an array of users subscriptions, which will consist of product codes like this:

$productCodes = ['aa1', 'bb2', 'cc3'];

I need to expand this query and sort the collections further by the $productCodes array that I will get. I need to check the codes from the productCodes array in the table magazines where I have the product_code field. The collections of issues grouped by magazine, should be then sorted so that the first collections are the ones whose magazine that they belong to has the same product_code as the code in the array productCodes, and amongst them, the first one would be whose collection has the latest issue by date. Then the rest of the collections should just be sorted by date. How can I make this kind of query?

Update

I have tried with a suggested code from @Paul Spiegel in the answers, and now I get an array of collections, with the collections of magazine issues. Issues in each magazine collection are ordered by date, and magazine collections that have the same product_code as in the $productCodes array are at the beginning of the array, but the the array of magazine collections is still not sorted by the date of the latest issue from each of the magazine collections.

Ludwig
  • 1,401
  • 13
  • 62
  • 125
  • `Issues::groupBy('magazine_id')->orderBy('date', 'desc')->orderBy('product_codes', 'desc')->get()`. Ordered by can be invoked as many times as you wish. – Andrei Nov 04 '16 at 08:43
  • But the problem is that `product_code` is not a field in the `issues` table it is in the relationship table `magazines` – Ludwig Nov 04 '16 at 08:45
  • Ah, i see. My bad. Well then: `Issues::groupBy('magazine_id')->with('magazine' => function($q){ $q->orderBy('product_codes', 'desc')->get(); } )->orderBy('date', 'desc')->get();`. My laravel skills are a bit rusty to say the least, but I think this should work. – Andrei Nov 04 '16 at 08:48
  • As I said, a bit rusty. Forgot that `with` needs to be an array in this case. `Issues::groupBy ('magazine_id')->with ([ 'magazine' => function ($q) { $q->orderBy ('product_codes', 'desc')->get (); } ])->orderBy ('date', 'desc')->get ();` – Andrei Nov 04 '16 at 08:53
  • Thanks for you effort, but I still get an error: `SQLSTATE[42000]: Syntax error or access violation: 1055 Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'aftenposten.issues.id' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by (SQL: select * from `issues` group by `magazine_id` order by `date` desc)` – Ludwig Nov 04 '16 at 08:57
  • Right, mysql `5.7.x` changed that. I don't think you can do it using eloquent. You'll have to use raw query. You'd need to specify all the colums you wish yo group by in a single statement, which is not possible using eloquent. Check out [this](http://stackoverflow.com/questions/37951742/1055-expression-of-select-list-is-not-in-group-by-clause-and-contains-nonaggr) question. Either that or change the mysql settings. – Andrei Nov 04 '16 at 09:02
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/127359/discussion-between-andrew-and-marco). – Andrei Nov 04 '16 at 09:04
  • You can turn off strict mode in `config/database.php` by addin `'strict' => false,` to the mysql connection array – Eric Tucker Nov 04 '16 at 17:24
  • @RyanVincent I am sorry if I wasn't clear enough, I want the output as I have it now, 1 collection array, with 24 collection arrays inside of it, as it is shown in the question. What I want to add to it, is to sort those 24 collections inside this main collection first by the ones whose items have the same product code as in the array productCodes and also by the date of the latest issue. Paul Spiegel has almost what I need, it is just missing sorting the collection array with the date of the latest issue inside of it. – Ludwig Nov 07 '16 at 19:53

3 Answers3

3

First of all: Instead of using Collection::groupBy() you can just use relations:

$magazines = Magazine::with(['issues' => function($query) {
    $query->orderBy('date', 'desc');
}]);

This will give you a collection of Magazines including related issues in an array. The issues array is sorted by date desc.

Now you can split the result using two queries with different conditions:

$baseQuery = Magazine::with(['issues' => function($query) {
    $query->orderBy('date', 'desc');
}]);

$query1 = clone $baseQuery;
$query2 = clone $baseQuery;

$query1->whereIn('product_code', $productCodes);
$query2->whereNotIn('product_code', $productCodes);

$query1 will return all magazines with product_code from the array. $query2 will return all the other magazines.

You have now two ways to combine the two results:

1) Use Eloquent::unionAll()

$unionQuery = $query1->unionAll($query2);
$magazines = $unionQuery->get();

2) Use Collection::merge()

$magazines1 = $query1->get();
$magazines2 = $query2->get();
$magazines = $magazines1->merge($magazines2);

In both cases you will get the same result: A collection of magazines. The magazines with product_code from the $productCodes array are sorted first. Every Magazine object contains an array issues with related Issue objects.

If you want to get rid of the Magazine objects and really need the result to look like in your question, you can still do:

$issues = $magazines->keyBy('id')->pluck('issue');

But there is probably no need for that.

Update

If you really need the two parts of the Magazine collection to be sorted by the latest Issue.. I don't see any way without using a JOIN or sorting in PHP with a closure. Sorting magazines with a join would also need an aggregation. So i would switch back to your original query extending it with a join and split it in two parts as showed above:

$baseQuery = Issue::join('magazines', 'magazines.id', '=', 'issues.magazine_id');
$baseQuery->orderBy('issues.date', 'desc');
$baseQuery->select('issues.*');

$query1 = clone $baseQuery;
$query2 = clone $baseQuery;

$query1->whereIn('magazines.product_code', $productCodes);
$query2->whereNotIn('magazines.product_code', $productCodes);

And then

$issues1 = $query1->get()->groupBy('magazine_id');
$issues2 = $query2->get()->groupBy('magazine_id');

$issues = $issues1->union($issues2);

or

$issues = $query1->unionAll($query2)->get()->groupBy('optionGroupId');
Paul Spiegel
  • 30,925
  • 5
  • 44
  • 53
  • This works for getting the magazines with the same product code first in the array, but the rest of the magazines that don't have the same product code are not ordered by date of their latest issue. – Ludwig Nov 07 '16 at 11:51
  • Just realized that neither the magazines that have the same product code as in the array product codes or the magazines that don't have the same product code are ordered by the date of their latest issue. – Ludwig Nov 07 '16 at 11:58
  • 1
    I hoped, that sorting by latest issue was not a requirement, but just happened :-) - I will look at it later. – Paul Spiegel Nov 07 '16 at 12:34
  • I have tried your updated suggestions but then I get: `Method getKey does not exist.` – Ludwig Nov 08 '16 at 08:26
  • Try `union()` instead of `merge()`. The keys of the two arrays are distinct, so there shouldn't be a difference. I have no clue, why `merge()` doesn't work here. – Paul Spiegel Nov 08 '16 at 13:25
  • This seems to be working, but I don't then the image property is empty on the issue object, is that because I have image field in both magazine and issues table? – Ludwig Nov 08 '16 at 13:53
  • Probably - yes. Add `$baseQuery->select('issues.*');` – Paul Spiegel Nov 08 '16 at 13:56
2

Code first, step-by-step explanation at the end:

Magazine::all()
    ->load('issues')
    ->keyBy('id')
    ->sort(function ($a, $b) use ($product_codes) {
        return
            (in_array($b->product_code, $product_codes) - in_array($a->product_code, $product_codes))
            ?:
            (strtotime($b->latestIssueDate) - strtotime($a->latestIssueDate));
    })
    ->map(function ($m) {
        return $m->issues->sortByDesc('created_at');
    });

This gets you:

  • all Magazines
  • with each of their Issues eager-loaded
  • in a Collection where the keys are each Magazine id
  • which is sorted by 2 criteria: first whether the product_code is among the requested codes, then the date of the latest Issue (*)
  • finally, for each Magazine you want a Collection of its issues, sorted by date descending.

If I understood your requirement correctly, this will give you what you want.

(This assumes the date column is called created_at.)


(*) for convenience I'm using an attribute accessor:

public function getLatestIssueDateAttribute()
{
    // this breaks for Magazines with no Issues. Fix if necessary
    return $this->issues->sortByDesc('created_at')->first()->created_at;
}
alepeino
  • 9,551
  • 3
  • 28
  • 48
  • @PaulSpiegel no, the `load` method will fetch all related models in 1 query. So 2 queries total – alepeino Nov 08 '16 at 14:33
  • I mean the `getLatestIssueDateAttribute()` method. – Paul Spiegel Nov 08 '16 at 14:47
  • I have just tested it and it works as well. Which method would you performance wise recommend yours or Pauls? – Ludwig Nov 08 '16 at 17:17
  • 1
    I'm not sure. I find my version easier to read. As for database optimization, this method only needs 2 queries (thanks to Paul's observation about my previous, suboptimal, version). I'd like to read @PaulSpiegel 's opinion on your question as well – alepeino Nov 08 '16 at 17:41
  • The update doesn't change the number of queries. The only difference is that the first version (`latest()`) only works with the `created_at` column. Put this line `\DB::listen(function($query) {echo "\n
    {$query->sql}
    \n";});` in front of your code - it will print every executed sql query.
    – Paul Spiegel Nov 08 '16 at 22:21
  • @PaulSpiegel It does. I had tested it with `DB::getQueryLog()`. This `$this->issues->sortBy...` operates on the collection of issues, which were previously eager loaded, it doesn't query the DB again. – alepeino Nov 09 '16 at 12:48
  • Sorry.. i've overseen this change: `->issues()->` => `->issues->`. However usually sorting in the db should be faster. But for small data sets it might not matter. – Paul Spiegel Nov 09 '16 at 13:42
1

First thing is with needs to be an array and when using a subquery, don't call get() as it will replicate the query for each Issue giving you a nice little N+1 problem. For the groupBy simply pass id as a 2nd parameter to help with the only_full_group_by error.

It should look like this:

$issues = Issue::groupBy('magazine_id', 'id')
    ->with('magazine')
    ->orderBy('date', 'desc')
    ->get();

That will order your Issue collection by date and then in each $issues->magazine will be ordered by product_code if you want to order this in one query by date and then by then product_code you need to do a join like:

$issues = Issue::select('issues.*')
    ->groupBy('magazine_id', 'id')
    ->join('magazine', 'magazine.id', '=', 'issues.magazine_id')
    ->with('magazine')
    ->orderBy('issues.date', 'desc')
    ->orderBy('magazine.product_code', $productCodes)
    ->get();

UPDATE

The following should give you grouped magazine collections sorted by date with each magazine collection sorted by product codes in the order specified in the $productCodes array:

$issues = Issue::groupBy('magazine_id', 'id')
    ->with('magazine')
    ->orderBy('date', 'desc')
    ->get()
    ->groupBy('magazine_id')
    ->sortBy(function($issue) use ($productCodes) {
        $index = array_search($issue->first()->magazine->product_id, $productCodes);

        return $index !== false ? $index : $issue->magazine_date;
    });
Eric Tucker
  • 6,144
  • 1
  • 22
  • 36
  • Thank you for your effort, but this query doesn't work for me. With my query I was getting an array with 24 collections of issues grouped by magazine id. Now I am getting all the issues in one array. I still need to get a collections of issues grouped by the magazine id, the only thing new that I need to do is to somehow sort those collections of issues and have the collections of issues whose magazine that they belong to has the same product_code as the codes in the productCodes array first in the array. – Ludwig Nov 04 '16 at 17:46
  • Whoops, didn't see that was an array. The updated query should work to sort by the given array – Eric Tucker Nov 04 '16 at 17:54
  • I still get all of the issues in one array, and not the array of collections of issues. I also see that I haven't made myself clear of what was I getting from my query. I have corrected that in my question. – Ludwig Nov 04 '16 at 17:57
  • I think I understand, does this last edit solve it? You want Issues grouped by magazine collections and ordered by date and each magazine collection ordered by product_code, correct? – Eric Tucker Nov 04 '16 at 18:13
  • This as it seems to me, sorts the issues inside of each collection by product_code, what I need is to somehow sort collections by product_code. – Ludwig Nov 04 '16 at 18:22
  • I have tried to further explain what I am trying to achieve here. – Ludwig Nov 04 '16 at 18:43
  • See if my last edit does the trick, it sort everything in `$productCodes` then by date after. – Eric Tucker Nov 04 '16 at 21:17
  • Unfortunately it doesn't work, I get an error for undefined variable $issue. – Ludwig Nov 05 '16 at 09:57
  • Whoops, edited. `$issue` should be injected into the `sortBy` callback. – Eric Tucker Nov 07 '16 at 17:04
  • Then I get `Undefined property: Illuminate\Database\Eloquent\Collection::$magazine` – Ludwig Nov 08 '16 at 08:19
  • Updated again, `$issues` is a collection instead of an object after the `groupBy` – Eric Tucker Nov 08 '16 at 17:51