PostgreSQL splitting time range into days

Question

I'm trying to write a complex query using PostgreSQL 9.2.4, and I'm having trouble getting it working. I have a table which contains a time range, as well as several other columns. When I store data in this table, if all of the columns are the same and the time ranges overlap or are adjacent, I combine them into one row.

When I retrieve them, though, I want to split the ranges at day boundaries - so for example:

2013-01-01 00:00:00 to 2013-01-02 23:59:59

would be selected as two rows:

2013-01-01 00:00:00 to 2013-01-01 23:59:59
2013-01-02 00:00:00 to 2013-01-02 23:59:59

with the values in the other columns the same for both retrieved entries.

I have seen this question which seems to more or less address what I want, but it's for a "very old" version of PostgreSQL, so I'm not sure it's really still applicable.

I've also seen this question, which does exactly what I want, but as far as I know the CONNECT BY statement is an Oracle extension to the SQL standard, so I can't use it.

I believe I can achieve this using PostgreSQL's generate_series, but I'm hoping there's a simple example out there demonstrating how it can be used to do this.

This is the query I'm working on at the moment, which currently doesn't work (because I can't reference the FROM table in a joined subquery), but I believe this is more-or-less the right track.

Here's the fiddle with the schema, sample data, and my working query.

Update: I just found out a fun fact, thanks to this question, that if you use a set-returning function in the SELECT part of the query, PostgreSQL will "automagically" do a cross join on the set and the row. I think I'm close to getting this working.

Should go without saying, your version of Postgres is essential to the question. How can one worry about code being "very old" and still keep that information from us? — Erwin Brandstetter, Oct 18 '13 at 18:18
You're right, that was a stupid omission. I am using PostgreSQL 9.2.4. — CmdrMoozy, Oct 18 '13 at 18:20
Best put that in your question instead of just the comment ... While being at it, also add the query you have right now, even if it's not working ... — Erwin Brandstetter, Oct 18 '13 at 18:21
Concerning SRF functions ("automagically" doing a cross join) in the target list: this is a very powerful feature, but can have nasty side effects and is totally non-SQL-standard. [Consider this related answer and the comments and links for more information](http://stackoverflow.com/questions/19059654/return-list-with-dates-within-a-specified-range-from-a-single-query/19059726#comment28176888_19059726)! The gist of it: once you have Postgres *9.3* use `LATERAL` instead - as detailed in my answer below. — Erwin Brandstetter, Oct 18 '13 at 20:09

score 13 · Accepted Answer · edited May 23 '17 at 12:10

First off, your upper border concept is broken. A timestamp with 23:59:59 is no good. The data type timestamp has fractional digits. What about 2013-10-18 23:59:59.123::timestamp?

Include the lower border and exclude the upper border everywhere in your logic. Compare:

Calculate number of concurrent events in SQL

Building on this premise:

Postgres 9.2 or older

SELECT id
     , stime
     , etime
FROM   timesheet_entries t
WHERE  etime <= stime::date + 1  -- this includes upper border 00:00

UNION ALL
SELECT id
     , CASE WHEN stime::date = d THEN stime ELSE d END     -- AS stime
     , CASE WHEN etime::date = d THEN etime ELSE d + 1 END -- AS etime
FROM (
   SELECT id
        , stime
        , etime
        , generate_series(stime::date, etime::date, interval '1d')::date AS d
   FROM   timesheet_entries t
   WHERE  etime > stime::date + 1
   ) sub
ORDER  BY id, stime;

Or simply:

SELECT id
     , CASE WHEN stime::date = d THEN stime ELSE d END     -- AS stime
     , CASE WHEN etime::date = d THEN etime ELSE d + 1 END -- AS etime
FROM (
   SELECT id
        , stime
        , etime
        , generate_series(stime::date, etime::date, interval '1d')::date AS d
   FROM   timesheet_entries t
   ) sub
ORDER  BY id, stime;

The simpler one may even be faster.
Note a corner case difference when stime and etime both fall on 00:00 exactly. Then a row with a zero time range is added at the end. There are various ways to deal with that. I propose:

SELECT *
FROM  (
   SELECT id
        , CASE WHEN stime::date = d THEN stime ELSE d END     AS stime
        , CASE WHEN etime::date = d THEN etime ELSE d + 1 END AS etime
   FROM (
      SELECT id
           , stime
           , etime
           , generate_series(stime::date, etime::date, interval '1d')::date AS d
      FROM   timesheet_entries t
      ) sub1
   ORDER  BY id, stime
   ) sub2
WHERE  etime <> stime;

Postgres 9.3+

In Postgres 9.3+ you would better use LATERAL for this

SELECT id
     , CASE WHEN stime::date = d THEN stime ELSE d END     AS stime
     , CASE WHEN etime::date = d THEN etime ELSE d + 1 END AS etime
FROM   timesheet_entries t
     , LATERAL (SELECT d::date
                FROM   generate_series(t.stime::date, t.etime::date, interval '1d') d
                ) d
ORDER  BY id, stime;

Details in the manual.
Same corner case as above.

SQL Fiddle demonstrating all.

This is an exceptionally awesome answer. It solves not only the problem I was having, but also a problem or two I didn't even know I had yet. Thanks! :) — CmdrMoozy, Oct 18 '13 at 19:29
@CmdrMoozy: There is a notable **corner case**. I added a fix for that. Also, I would be interested how the query compares to [Pavel's function](http://stackoverflow.com/a/19456611/939860). If you happen to test both, you might leave a note with the results. — Erwin Brandstetter, Oct 18 '13 at 19:55
One small correction: in the 3rd query in your post, I believe `generate_series(stime, etime, interval '1d')::date` should be `generate_series(stime::date, etime::date, interval '1d')` instead, to handle the case where `stime` and `etime` fall on different days, but are less than one day apart. — CmdrMoozy, Oct 18 '13 at 22:39
@CmdrMoozy: Right, that should make it complete. Applied your fix. — Erwin Brandstetter, Oct 18 '13 at 23:45

Pavel Stehule · Answer 2 · 2013-10-18T20:45:29.190

There is simply solution (if intervals starts in same time)

postgres=# select i, i + interval '1day' - interval '1sec' 
  from generate_series('2013-01-01 00:00:00'::timestamp, '2013-01-02 23:59:59', '1day') g(i);
          i          │      ?column?       
─────────────────────┼─────────────────────
 2013-01-01 00:00:00 │ 2013-01-01 23:59:59
 2013-01-02 00:00:00 │ 2013-01-02 23:59:59
(2 rows)

I wrote a table function, that do it for any interval. It is fast - two years range divide to 753 ranges in 10ms

create or replace function day_ranges(timestamp, timestamp)
returns table(t1 timestamp, t2 timestamp) as $$
begin
  t1 := $1;
  if $2 > $1 then
    loop
      if t1::date = $2::date then
        t2 := $2;
        return next;
        exit;
      end if;
      t2 := date_trunc('day', t1) + interval '1day' - interval '1sec';
      return next;
      t1 := t2 + interval '1sec';
    end loop;
  end if;
  return;
end;
$$ language plpgsql;

Result:

postgres=# select * from day_ranges('2013-10-08 22:00:00', '2013-10-10 23:00:00');
         t1          │         t2          
─────────────────────┼─────────────────────
 2013-10-08 22:00:00 │ 2013-10-09 23:59:59
 2013-10-09 00:00:00 │ 2013-10-09 23:59:59
 2013-10-10 00:00:00 │ 2013-10-10 23:00:00
(3 rows)

Time: 6.794 ms

and faster (and little bit longer) version based on RETURN QUERY

create or replace function day_ranges(timestamp, timestamp)
returns table(t1 timestamp, t2 timestamp) as $$
begin
  t1 := $1; t2 := $2;
  if $1::date = $2::date then
    return next;
  else
    -- first day
    t2 := date_trunc('day', t1) + interval '1day' - interval '1sec';
    return next;
    if $2::date > $1::date + 1 then
      return query select d, d + interval '1day' - interval '1sec'
                      from generate_series(date_trunc('day', $1 + interval '1day')::timestamp,
                                           date_trunc('day', $2 - interval '1day')::timestamp,
                                           '1day') g(d);
    end if;
    -- last day 
    t1 := date_trunc('day', $2); t2 := $2;
    return next;
  end if;
  return;
end;
$$ language plpgsql;

In my case, the "from" and "to" times are arbitrary, although I have a constraint which guarantees `from < to`. The other problem is that I need to select some other columns besides `from` and `to`, copying the values of the other columns for each "split" result row. — CmdrMoozy, Oct 18 '13 at 18:28
@CmdrMoozy: It becomes obvious that you also should provide come sample data and probably also the table definition. Best in a SQLfiddle. Random example: http://www.sqlfiddle.com/#!12/1cb5c/8 — Erwin Brandstetter, Oct 18 '13 at 18:31

score 1 · Answer 3 · edited May 23 '17 at 11:54

You may build the result by simply generating all days in the entire set of ranges and join this to the ranges themselves using the OVERLAPS operator. This will both discard the days for which there is no data and do the required cartesian product for the days with data.

OVERLAPS behavior is explained in the doc as this:

(start1, end1) OVERLAPS (start2, end2)
(start1, length1) OVERLAPS (start2, length2)
This expression yields true when two time periods (defined by their endpoints) overlap, false when they do not overlap. The endpoints can be specified as pairs of dates, times, or time stamps; or as a date, time, or time stamp followed by an interval. When a pair of values is provided, either the start or the end can be written first; OVERLAPS automatically takes the earlier value of the pair as the start. Each time period is considered to represent the half-open interval start <= time < end, unless start and end are equal in which case it represents that single time instant. This means for instance that two time periods with only an endpoint in common do not overlap.

With your timesheet_entries table, the query would be:

select days.day, timesheet_entries.* from
  (select day from generate_series(
         (select min(stime) from timesheet_entries),
         (select max(etime) from timesheet_entries),
         '1 day'::interval) day
  ) days
 join timesheet_entries 
  on (stime,etime) overlaps (days.day,days.day+'1 day'::interval)
 order by 1;

This technique makes it also easy to add the days without data, calendar-like. To do so, just replace the join by a left join.

As pointed by @Erwin in his answer, etime should not represent the last second (...:59:59) of the interval but the next second as an excluded upper bound. It may not matter with your current data, but if certain rows had stime at ...:59:59 or if you happened to need sub-second resolution, that would be a problem.

PostgreSQL splitting time range into days

3 Answers3

Postgres 9.2 or older

Postgres 9.3+