0

This each statement is taking 1+ seconds with around 20000 elements, but it doesn't seem like it should take that long at first glance. The time must come from the field lookup for ping and holder. Aside from pulling holder.created_at out to reduce lookups, is there anything I can do to make this faster?

This code block is given a list from the database of all elements created in the last X time, and in this case I want to get one data point for roughly each 6 minutes.

I do need to check each data point because I will eventually do some more work within the each block that will require looking at each data point, so I can't pull less information from the database.

temp = @modem.pings.where('created_at >= ?', 30.days.ago)
holder = Temp.first
@pings = Array.new

temp.each do |ping|
  if (ping.created_at - holder.created_at >= 21600)
    @pings << ping
    holder = ping
  end
end

Edit: I've removed the holder.created_at call from the loop which should reduce the number of lookups by a little less than half, but there has been no decrease in the response time. Am I looking in the wrong place? Does the temp.each do |ping| line have a high cost such that running it 20000 times takes a while?

Edit2: Learned about code profiling. The actual sql statement doesn't seem to take long. If I run the command and don't do anything with the returned data, it is very fast. But if I do something with the data it returns then I spend about 1/3 of my time in ActiveRecord::AssociationRelation#exec_queries. After exec_queries finishes I seem to spend a lot of the rest of my time in Time related functions:

Stack trace during time functions

Is there a way that I can avoid doing Time stuff? I believe created_at is stored as an integer, so is there a way to get the number and use that rather than have rails convert it a bunch of times?

John Moffitt
  • 5,730
  • 7
  • 30
  • 39
  • Is there a reason why you cannot accomplish this with SQL and a window `LAG`? Are you in Postgres? – snkashis Jun 15 '16 at 18:33
  • @snkashis Because I am new to Rails and do not know what a window `LAG` is. For the SQL part I'll be doing additional processing on the data shortly to drop or skip some datapoints based on some user-set variables that I am unsure if it can be done in SQL. I'm currently on sqlite but will be moving to Postgres soon. – John Moffitt Jun 15 '16 at 18:46
  • Have you run it through the profiler to spot what the hotspots are? – Frederick Cheung Jun 17 '16 at 20:19
  • Rails is smart enough to only make the SQL call once and cache it when using holder.created_at in that context. That's why you aren't seeing a big performance increase by moving that outside the loop (unlike, say, JavaScript which will re-initialize the variable each time you loop). – hightempo Jun 18 '16 at 18:17
  • Actually a lag may not be needed here, you may want to look into this http://stackoverflow.com/a/8194088/1238737 – snkashis Jun 21 '16 at 17:19

0 Answers0