108

Ok, this is my query:

SELECT
  video_category,
  video_url,
  video_date,
  video_title,
  short_description,
  MAX(video_id) 
FROM
  videos
GROUP BY
  video_category

When it pulls the data, I get the correct row for the video_id, but it pulls the first row for each category for the others. So when I get the max result for the video_id of category 1, I get the max ID, but the first row in the table for the url, date, title, and description.

How can I have it pull the other columns that correspond with the max ID result?

Edit: Fixed.

SELECT
    *
FROM
    videos
WHERE
    video_id IN
    (
        SELECT
            DISTINCT
            MAX(video_id)
        FROM
            videos
        GROUP BY
            video_category
    ) 
ORDER BY
    video_category ASC
Steven Moseley
  • 15,871
  • 4
  • 39
  • 50
Devin
  • 2,146
  • 5
  • 24
  • 36

5 Answers5

75

I would try something like this:

SELECT
   s.video_id
   ,s.video_category
   ,s.video_url
   ,s.video_date
   ,s.video_title
   ,s.short_description
FROM videos s
   JOIN (SELECT MAX(video_id) AS id FROM videos GROUP BY video_category) max
      ON s.video_id = max.id

which is quite a lot faster than your own solution

Community
  • 1
  • 1
Dalen
  • 8,856
  • 4
  • 47
  • 52
41

I recently invented a new technique to handle this type of problem in MySQL.

SCALAR-AGGREGATE REDUCTION

Scalar-Aggregate Reduction is by far the highest-performance approach and simplest method (in DB engine terms) for accomplishing this, because it requires no joins, no subqueries, and no CTE.

For your query, it would look something like this:

SELECT
  video_category,
  MAX(video_id) AS video_id,
  SUBSTRING(MAX(CONCAT(LPAD(video_id, 11, '0'), video_url)), 12) AS video_url,
  SUBSTRING(MAX(CONCAT(LPAD(video_id, 11, '0'), video_date)), 12) AS video_date,
  SUBSTRING(MAX(CONCAT(LPAD(video_id, 11, '0'), video_title)), 12) AS video_title,
  SUBSTRING(MAX(CONCAT(LPAD(video_id, 11, '0'), short_description)), 12) AS short_description
FROM
  videos
GROUP BY
  video_category

The combination of scalar and aggregate functions does the following:

  1. LPADs the intra-aggregate correlated identifier to allow proper string comparison (e.g. "0009" and "0025" will be properly ranked). I'm LPADDING to 11 characters here assuming an INT primary key. If you use a BIGINT, you will want to increase this to support your table's ordinality. If you're comparing on a DATETIME field (fixed length), no padding is necessary.
  2. CONCATs the padded identifier with the output column (so you get "00000000009myvalue" vs "0000000025othervalue")
  3. MAX the aggregate set, which will yield "00000000025othervalue" as the winner.
  4. SUBSTRING the result, which will truncate the compared identifier portion, leaving only the value.

If you want to retrieve values in types other than CHAR, you may need to performa an additional CAST on the output, e.g. if you want video_date to be a DATETIME:

CAST(SUBSTRING(MAX(CONCAT(LPAD(video_id, 11, '0'), video_date)), 12) AS DATETIME)

Another benefit of this method over the self-joining method is that you can combine other aggregate data (not just latest values), or even combine first AND last item in the same query, e.g.

SELECT
    -- Overall totals
    video_category,
    COUNT(1) AS videos_in_category,
    DATEDIFF(MAX(video_date), MIN(video_date)) AS timespan,
    
    -- Last video details
    MAX(video_id) AS last_video_id,
    SUBSTRING(MAX(CONCAT(LPAD(video_id, 11, '0'), video_url)), 12) AS last_video_url,
    ...
    
    -- First video details
    MIN(video_id) AS first_video_id,
    SUBSTRING(MIN(CONCAT(LPAD(video_id, 11, '0'), video_url)), 12) AS first_video_url,
    ...
    
    -- And so on

For further details explaining the benefits of this method vs other older methods, my full blog post is here: https://www.stevenmoseley.com/blog/tech/high-performance-sql-correlated-scalar-aggregate-reduction-queries

Steven Moseley
  • 15,871
  • 4
  • 39
  • 50
  • 4
    FYI, benchmarked on a 1-million row production table in Aurora, Scalar Aggregate Comparison performed 20% better than the Subquery method. – Steven Moseley Nov 06 '19 at 23:18
  • 5
    This is amazing, and EXACTLY what I was looking for. Thank you for this. I look forward to finding additional use cases for this method. – Mzril Dec 03 '19 at 23:53
  • 2
    I just spent a few hours optimizing a similar query on SQL Server 2014 and was coming on here to report the performance improvements that can be gained from this technique but found that you beat me to it. In my case, I was looking for the first & last date in an activity log along with the person who performed it. The T-SQL expressions I used to find the first action date and person were MIN(ActivityDate) FirstActivityDate and SUBSTRING(MIN(CONVERT(VARCHAR, ActivityDate, 21) + ActivityBy), 24, 256) FirstActivityBy. I got a 3X performance improvement over the CTE method for 130,000 rows. – ARM07470 Dec 04 '19 at 04:43
  • Great, now can someone write a LINQ query or method syntax version of this? :D – jamheadart Sep 08 '20 at 09:37
  • 3
    How do we get this baked into MySQL? I shouldn’t have to do this myself. – tuxedobob Mar 26 '21 at 20:13
  • @tuxedobob I was thinking this would be nice if written as an aggregate UDF, but that would require one dust off their C skills. :) – Steven Moseley Mar 26 '21 at 22:14
  • Can you please explain why you LPAD to a width of 11 for an INT type? The maximum value of an unsigned 32 bit int is only 10 digits – pigi5 Aug 28 '21 at 06:27
  • @pigi5 INT is 11 character max (negative sign). UNSIGNED INT is 10. As I mentioned in the post, you need to set that number to your app's ordinality. – Steven Moseley Sep 02 '21 at 13:57
  • 1
    Hah of course, I forgot about the negative sign. Thanks for the great solution! – pigi5 Sep 03 '21 at 14:32
  • 1
    This is awesome. I often need to get the latest event and then join to other tables using the event guid, this saves a costly join, and can use like this, eg. for each employee number: "SELECT MAX(ClockDateTime) AS LastClock, REPLACE(MAX(CONCAT(CONVERT(int,ClockDateTime),'|',se.ClockEventGuid)),CONCAT(CONVERT(int,MAX(ClockDateTime)),'|'),'') AS LastEventGuid, EmployeeNumber" – onemorecupofcoffee Apr 26 '23 at 06:59
  • 1
    No clue how this works but thank you. Will use – Ares Stavropoulos May 12 '23 at 21:49
7

A slightly more "rustic" solution, but should do the job just the same:

SELECT
  video_category,
  video_url,
  video_date,
  video_title,
  short_description,
  video_id
FROM
  videos
ORDER BY video_id DESC
LIMIT 1;

In other words, just produce a table with all of the columns that you want, sort it so that your maximum value is at the top, and chop it off so you only return one row.

Lou
  • 2,200
  • 2
  • 33
  • 66
6

Here is a more general solution (handles duplicates)

CREATE TABLE test(
  i INTEGER,
  c INTEGER,
  v INTEGER
);


insert into test(i, c, v)
values
(3, 1, 1),
(3, 2, 2),
(3, 3, 3),
(4, 2, 4),
(4, 3, 5),
(4, 4, 6),
(5, 3, 7),
(5, 4, 8),
(5, 5, 9),
(6, 4, 10),
(6, 5, 11),
(6, 6, 12);



SELECT t.c, t.v
FROM test t
JOIN (SELECT test.c, max(i) as mi FROM test GROUP BY c) j ON
  t.i = j.mi AND
  t.c  = j.c
ORDER BY c;
Guillaume Massé
  • 8,004
  • 8
  • 44
  • 57
-1

SELECT video_category,video_url,video_date,video_title,short_description,video_id FROM videos t1 where video_id in (SELECT max(video_id) FROM videos t2 WHERE t1.video_category=t2.video_category );

Please provide your input and output records so that it can be understood properly and tested.

  • 2
    This does not provide an answer to the question. Once you have sufficient [reputation](/help/whats-reputation) you will be able to [comment on any post](/help/privileges/comment); instead, [provide answers that don't require clarification from the asker](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead). – Yunnosch Nov 18 '20 at 20:43