MYSQL Query very slow when using ORDER BY datetime

Question

I have the following query which runs on a social network. The query fetches posts (like Facebook posts) from a database.

SELECT P.*, 
   P.id_post id_p, 
   PM.meta_content video_title, 
   PM2.meta_content video_views, 
   PM3.meta_content racebooking_views, 
   Greatest(P.creation_date, Coalesce(Max(C.date), P.creation_date)) AS 
   last_activity, 
   P.creation_date creation_date, 
   (SELECT Count(*) 
    FROM   likes 
    WHERE  post_id = P.id_post 
           AND post_type = 'P') 
   likes_count, 
   (SELECT Count(*) 
    FROM   likes L 
    WHERE  post_id = P.id_post 
           AND post_type = 'P' 
           AND L.id_profile = 2796) 
   do_i_like 
FROM   posts P 
       LEFT JOIN comments C 
              ON P.id_post = C.post_id 
                 AND C.post_type = 'P' 
                 AND C.id_profile != P.id_profile 
       LEFT JOIN post_meta PM 
              ON PM.id_post = P.id_post 
                 AND PM.meta_type = 'T' 
       LEFT JOIN post_meta PM2 
              ON PM2.id_post = P.id_post 
                 AND PM2.meta_type = 'V' 
       LEFT JOIN post_meta PM3 
              ON PM3.id_post = P.id_post 
                 AND PM3.meta_type = 'W' 
GROUP  BY P.id_post 
ORDER  BY last_activity DESC 
LIMIT  41, 10

Each post may have or may not have comments. I want the query to fetch the post with the most recent activity first. So, if the post has a comment, i take the date of the latest comment. If the post does not have a comment, i take the creation date of the post.

The job is done by Greatest(P.creation_date, Coalesce(Max(C.date), P.creation_date)) which picks up the greates value between the comments dates (if comments exist) and the post creation date.

Then, the ORDER BY last_activity DESC does the sorting job.

PROBLEM

The query is really slow. It takes 8 seconds to run. The posts table has 8K rows and the comments table has 8K rows.

What i don't understand is that if I replace the ORDER BY clause with this ORDER BY P.id_post it takes 0.5 seconds to run. But if I replace the ORDER BY clause with ORDER BY P.creation_date again it takes 8 seconds. It seems that it doesn't like dates...

Additional infos

posts table has an index on creation_date.
comments table has an index on date
server runs LAMP on CentOS Linux 6.6
I tried other solutions on SO like this one but they didn't work

How can i fix this query to run faster?

@YuJiaao In the question i wrote "posts table has an index on creation_date". Do you mean that or something else? — Alberto Fontana, Oct 12 '17 at 00:06
You use group by `last_activity`, but last_activity is dynamically generated, I think that index does not help in such case, maybe add `last_activity` as a column for posts table? — Yu Jiaao, Oct 12 '17 at 00:12

Tim Biegeleisen · Answer 1 · 2017-10-12T01:26:29.570

The correlated subqueries in the select clause are probably killing you. Instead, join to a subquery which computes likes statistics:

SELECT P.*, 
    P.id_post id_p, 
    PM.meta_content video_title, 
    PM2.meta_content video_views, 
    PM3.meta_content racebooking_views, 
    GREATEST(P.creation_date, COALESCE(MAX(C.date), P.creation_date)) AS last_activity, 
    P.creation_date creation_date,
    t.likes_count,
    t.do_i_like
FROM posts P
LEFT JOIN
(
    SELECT
        post_id,
        SUM(CASE WHEN post_type = 'P' THEN 1 ELSE 0 END) AS likes_count,
        SUM(CASE WHEN post_type = 'P' AND L.id_profile = 2796
                 THEN 1 ELSE 0 END) AS do_i_like
    FROM likes
    GROUP BY post_id
) t
    ON t.post_id = P.id_post 
LEFT JOIN comments C 
    ON P.id_post = C.post_id AND
       C.post_type = 'P' AND
       C.id_profile != P.id_profile 
LEFT JOIN post_meta PM 
    ON PM.id_post = P.id_post AND
       PM.meta_type = 'T' 
LEFT JOIN post_meta PM2 
    ON PM2.id_post = P.id_post AND
       PM2.meta_type = 'V' 
LEFT JOIN post_meta PM3 
    ON PM3.id_post = P.id_post AND 
       PM3.meta_type = 'W' 
ORDER BY
    last_activity DESC 
LIMIT 41, 10

Also after editing your query I do not see a reason to be using GROUP BY in the outer query, so I removed it. And you should be using indices where appropriate, though my hunch is that my suggestion alone should give a noticeable performance boost.

Thanks for the answer! For now i used a horrible workaround just to make the website working. On Tuesday i will test your solution and give you a feedback! — Alberto Fontana, Oct 12 '17 at 11:58

score 0 · Answer 2 · answered Oct 12 '17 at 05:02

There is a MAX(C.Date) that would require a group by clause, however it too could be substituted for a subquery I believe:

SELECT P.*, 
    P.id_post id_p, 
    PM.meta_content video_title, 
    PM2.meta_content video_views, 
    PM3.meta_content racebooking_views, 
    GREATEST(P.creation_date, COALESCE(max_c_date, P.creation_date)) AS last_activity, 
    P.creation_date creation_date,
    t.likes_count,
    t.do_i_like
FROM posts P
LEFT JOIN
(
    SELECT
        post_id,
        SUM(CASE WHEN post_type = 'P' THEN 1 ELSE 0 END) AS likes_count,
        SUM(CASE WHEN post_type = 'P' AND L.id_profile = 2796
                 THEN 1 ELSE 0 END) AS do_i_like
    FROM likes
    GROUP BY post_id
) t
    ON t.post_id = P.id_post 
LEFT JOIN (
      SELECT
        comments.post_id,
        MAX(comments.date) max_c_date
      FROM comments
      inner join posts ON comments.post_id = posts.id_post
      where  comments.post_type = 'P' AND
             comments.id_profile != posts.id_profile 
      GROUP BY comments.post_id
) C 
    ON P.id_post = C.post_id AND
LEFT JOIN post_meta PM 
    ON PM.id_post = P.id_post AND
       PM.meta_type = 'T' 
LEFT JOIN post_meta PM2 
    ON PM2.id_post = P.id_post AND
       PM2.meta_type = 'V' 
LEFT JOIN post_meta PM3 
    ON PM3.id_post = P.id_post AND 
       PM3.meta_type = 'W' 
ORDER BY
    last_activity DESC 
LIMIT 41, 10

MYSQL Query very slow when using ORDER BY datetime

2 Answers2