SQL: Reporting on data by date

Question

I have a table I am trying to generate reports from. Its basically a log of when something breaks (goes down), and then gets fixed.

Table schema and some example data is below. To illustrate:
1 row is inserted when it goes down
1 row is inserted when it comes back up.

What I am trying to do is report on various aspects, things like:
Amount of downtime in a given day / week / month
Number of times its gone down in a given day / week / month.

Ideally in a way that would easily be exported to excel or something similar to be graphed.

I'm having trouble coming up with any kind of queries to get this info.

I have this one for example:

SELECT [Name], datepart(day,[Inserted]), count([SystemDown])     
FROM [DownTimeLog]
WHERE [SystemDown]=1
GROUP BY [Name],datepart(day,[Inserted])

which gives me the number of times the system has gone down per day, which is a good starting point.

But i'm trying to come up with a way of showing the total time its been down, but I am drawing a blank. Some days for example may be 0, sometimes it may go down a few times, so trying to sum the time difference between 2 corresponding rows is provinh tough.

CREATE TABLE [dbo].[DownTimeLog]
(
    [ID] [int] IDENTITY(1,1) NOT NULL,
    [Name] VARCHAR(30) NOT NULL,
    [SystemDown] BIT NOT NULL,
    [Inserted] DATETIME NOT NULL
)

INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 14 2011  1:49:58:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 14 2011  2:49:58:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 15 2011  1:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 15 2011  2:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 15 2011  4:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 15 2011  5:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 17 2011  1:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 17 2011  3:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 18 2011 10:00:00:000AM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 18 2011 11:00:00:000AM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 18 2011  1:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 18 2011  3:30:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 18 2011  4:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 18 2011  8:00:00:000PM')

So for example using the data above I'd like to pull data back that was something like:

I hope someone can give me a method of doing what I am trying to do.

-- Edit:

Thanks all for some great answers. Helped me out a tonne. I thought my sql was pretty string, never heard of a cross apply though - guess i need to go back to school!

Cheers!

What version of SQL server are you using? Something like [this self-join solution](http://stackoverflow.com/questions/5901418/sql-to-determine-multiple-date-ranges-sql-server-2000/5936686#5936686) would probably work for you. — Kit Z. Fox, Jun 20 '11 at 21:12
If the system is down from 11pm to 1am is that 1 hour in each day, or 2 hours in the first day? — MatBailie, Jun 20 '11 at 21:25
If the system is not down at all for a specific day, do you still need a record showing 0 hours? — MatBailie, Jun 20 '11 at 21:25

NullRef · Answer 1 · 2011-06-20T22:07:38.083

I think if you break these into two pseudo-tables you will be able to do more. Try this:

SELECT
    *
FROM
    ( SELECT
        *
      FROM
        [DownTimeLog] AS down
      WHERE
        [SystemDown] = 1
    ) down
    INNER JOIN ( SELECT
                    *
                 FROM
                    [DownTimeLog]
                 WHERE
                    SystemDown = 0
               ) up
    ON down.id = ( up.id - 1 ) and down.name = up.name
    ORDER BY down.[Inserted], up.inserted

You can then do all kinds of calculations on down records vs. the subsequent up records.

EDIT: Of course and @Dems points out this assumes IDs are sequential. If you would rather use dates as the filter do this:

SELECT
    *
FROM
    ( SELECT
        *
      FROM
        [DownTimeLog] AS down
      WHERE
        [SystemDown] = 1
    ) down
    CROSS APPLY ( SELECT TOP 1
                    *
                 FROM
                    [DownTimeLog] up
                 WHERE
                    SystemDown = 0

                    AND up.INSERTED > down.INSERTED
                    AND up.NAME = down.name
                    ORDER BY up.inserted 
               ) up
    ORDER BY down.[Inserted], up.inserted

Here is a more detailed example of what can be done:

SELECT
        [Down_Id]
    ,   [Down_Name]
    ,   [Down_SystemDown]
    ,   [Down_Inserted]
    ,   [Up_Id]
    ,   [Up_Name]
    ,   [Up_SystemDown]
    ,   [Up_Inserted]
    ,   CAST(DATEDIFF(mi,Down_Inserted,Up_Inserted) AS DECIMAL)/60  AS Hours_Down
    ,   DATEDIFF(mi,Down_Inserted,Up_Inserted)  AS Minutes_Down
FROM
    ( SELECT
        [ID] AS Down_Id
      , [Name] AS Down_Name
      , [SystemDown] AS Down_SystemDown
      , [Inserted] AS Down_Inserted
      FROM
        [DownTimeLog] AS down
      WHERE
        [SystemDown] = 1
    ) down
    CROSS APPLY ( SELECT TOP 1
                    [ID] AS Up_Id
                  , [Name] AS Up_Name
                  , [SystemDown] AS Up_SystemDown
                  , [Inserted] AS Up_Inserted
                  FROM
                    [DownTimeLog] up
                  WHERE
                    SystemDown = 0
                    AND up.[Inserted] > down.Down_Inserted
                    AND up.NAME = down.Down_name
                  ORDER BY
                    up.Inserted
                ) up
ORDER BY
    down.[Down_Inserted]
,   up.up_inserted

You've made the assumption that here is only one system inserting into the table, therefor all records are sequential. The presence of a System field leads me to make the opposite assumption. — MatBailie, Jun 20 '11 at 21:37
@Dems, you make a good point. I forgot to join on `name` at first. Also, depending on how the system is really setup it may be better to use the `inserted` column rather than rely on sequential IDs. — NullRef, Jun 20 '11 at 22:09

Mikael Eriksson · Answer 2 · 2011-06-20T21:51:24.760

Something like this might work for you.

select D1.Name,
       dateadd(d, datediff(d, 0, D1.Inserted), 0) as [Date],
       sum(datediff(mi, D1.Inserted, D2.Inserted)) as DownTime,
       count(*) as Occurrences
from DownTimeLog as D1
  cross apply ( select top 1 Name,
                             Inserted
                from DownTimeLog
                where Name = D1.Name and
                      Inserted > D1.Inserted 
                order by Inserted             
              ) as D2
where D1.SystemDown = 1
group by D1.Name, dateadd(d, datediff(d, 0, D1.Inserted), 0)

Result:

Name                           Date                    DownTime    Occurrences
------------------------------ ----------------------- ----------- -----------
System1                        2011-06-14 00:00:00.000 60          1
System1                        2011-06-15 00:00:00.000 120         2
System1                        2011-06-17 00:00:00.000 120         1
System1                        2011-06-18 00:00:00.000 450         3

You won't get a row for days with 0 occurrences. And if you have downtime over midnight all time will be counted on the day of the down event. DownTime is in minutes.

score 1 · Answer 3 · answered Jun 20 '11 at 22:09

WITH
  enhanced_log
AS
(
SELECT
  [start].Name                                              AS [Name],
  DATEADD(DAY, DATEDIFF(DAY, 0, [start].Inserted), 0)       AS [Date],
  [start].Inserted                                          AS [Start],
  [finish].Inserted                                         AS [Finish]
FROM
  DownTimeLog  AS [start]
OUTER APPLY
  (SELECT TOP 1 * FROM DownTimeLog WHERE name = [start].name AND Inserted > [start].Inserted ORDER BY Inserted ASC) AS [finish]
WHERE
  [start].SystemDown = 1

UNION ALL

SELECT
  [start].Name,
  [Calendar].date,
  [Calendar].date,
  [finish].Inserted
FROM
  calendar
CROSS JOIN
  system
CROSS APPLY
  (SELECT TOP 1 * FROM DownTimeLog WHERE name = [system].name AND Inserted < [calendar].date ORDER BY Inserted ASC) AS [start]
OUTER APPLY
  (SELECT TOP 1 * FROM DownTimeLog WHERE name = [system].name AND Inserted > [start].Inserted ORDER BY Inserted ASC) AS [finish]
WHERE
  [start].SystemDown = 1
)

SELECT
  Name,
  Date,
  SUM(DATEDIFF(MINUTE, Start, CASE WHEN Finish < Date + 1 THEN Finish ELSE Date + 1 END)) AS Duration,
  COUNT(*) AS Instances
FROM
  enhanced_log
WHERE
  Start <> CASE WHEN Finish < Date + 1 THEN Finish ELSE Date + 1 END
GROUP BY
  Name,
  Date

score 1 · Accepted Answer · answered Jun 20 '11 at 23:02

The following will produce your exact output including missing days. Note, this code is based on Mikael Eriksson's answer.

CREATE TABLE #DownTimeLog
(
    [ID] [int] IDENTITY(1,1) NOT NULL,
    [Name] VARCHAR(30) NOT NULL,
    [SystemDown] BIT NOT NULL,
    [Inserted] DATETIME NOT NULL
)

INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 14 2011  1:49:58:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 14 2011  2:49:58:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 15 2011  1:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 15 2011  2:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 15 2011  4:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 15 2011  5:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 17 2011  1:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 17 2011  3:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 18 2011 10:00:00:000AM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 18 2011 11:00:00:000AM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 18 2011  1:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 18 2011  3:30:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 18 2011  4:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 18 2011  8:00:00:000PM')

CREATE TABLE #DownTimeLogModified (Name nvarchar(512), [Date] nvarchar(512), DownTime int,  Occurrences int)

INSERT INTO #DownTimeLogModified (Name, [Date], DownTime,  Occurrences)
select D1.Name,
       CONVERT(VARCHAR(12), dateadd(d, datediff(d, 0, D1.Inserted), 0), 107) as [Date],
       sum(datediff(mi, D1.Inserted, D2.Inserted)) as DownTime,
       count(*) as Occurrences
from #DownTimeLog as D1
  cross apply ( select top 1 Name,
                             Inserted
                from #DownTimeLog
                where SystemDown = 0 and
                      Name = D1.Name and
                      Inserted > D1.Inserted 
                order by Inserted             
              ) as D2
where D1.SystemDown = 1
group by D1.Name, dateadd(d, datediff(d, 0, D1.Inserted), 0)

DECLARE @startdate datetime,@enddate datetime
select @startdate = MIN(Inserted) FROM #DownTimeLog
select @enddate = MAX(Inserted) FROM #DownTimeLog

;WITH DateIntervalsCTE AS
        (
        SELECT 1 i,@startdate AS Date
        UNION ALL
        SELECT i + 1, DATEADD(day, i, @startdate )
        FROM DateIntervalsCTE 
        WHERE DATEADD(day, i, @startdate ) <= @enddate
        )

SELECT CASE WHEN a.Name is null THEN 'No System downtime' ELSE a.Name END as Name,CONVERT(VARCHAR(12), b.Date, 107) AS Date,CASE WHEN a.DownTime is null THEN 0 ELSE a.DownTime END AS DownTime,CASE WHEN a.Occurrences is null THEN 0 ELSE a.Occurrences END AS Occurrences 
FROM DateIntervalsCTE b
LEFT JOIN #DownTimeLogModified a ON a.Date = CONVERT(VARCHAR(12), b.Date, 107)

DROP TABLE #DownTimeLog
DROP TABLE #DownTimeLogModified

Output

Name                           Date                    DownTime    Occurrences
------------------------------ ----------------------- ----------- -----------
System1                        Jun 14, 2011                60           1
System1                        Jun 15, 2011               120           2
No system downtime             Jun 16, 2011                 0           0
System1                        Jun 17, 2011               120           1
System1                        Jun 18, 2011               450           3

SQL: Reporting on data by date

4 Answers4