Query without WHILE Loop

Question

We have appointment table as shown below. Each appointment need to be categorized as "New" or "Followup". Any appointment (for a patient) within 30 days of first appointment (of that patient) is Followup. After 30 days, appointment is again "New". Any appointment within 30 days become "Followup".

I am currently doing this by typing while loop.
How to achieve this without WHILE loop?

Table

CREATE TABLE #Appt1 (ApptID INT, PatientID INT, ApptDate DATE)
INSERT INTO #Appt1
SELECT  1,101,'2020-01-05' UNION
SELECT  2,505,'2020-01-06' UNION
SELECT  3,505,'2020-01-10' UNION
SELECT  4,505,'2020-01-20' UNION
SELECT  5,101,'2020-01-25' UNION
SELECT  6,101,'2020-02-12'  UNION
SELECT  7,101,'2020-02-20'  UNION
SELECT  8,101,'2020-03-30'  UNION
SELECT  9,303,'2020-01-28' UNION
SELECT  10,303,'2020-02-02' 

Answer 1

You need to use recursive query.

The 30days period is counted starting from prev(and no it is not possible to do it without recursion/quirky update/loop). That is why all the existing answer using only ROW_NUMBER failed.

WITH f AS (
  SELECT *, rn = ROW_NUMBER() OVER(PARTITION BY PatientId ORDER BY ApptDate) 
  FROM Appt1
), rec AS (
  SELECT Category = CAST('New' AS NVARCHAR(20)), ApptId, PatientId, ApptDate, rn, startDate = ApptDate
  FROM f
  WHERE rn = 1
  UNION ALL
  SELECT CAST(CASE WHEN DATEDIFF(DAY,  rec.startDate,f.ApptDate) <= 30 THEN N'FollowUp' ELSE N'New' END AS NVARCHAR(20)), 
         f.ApptId,f.PatientId,f.ApptDate, f.rn,
         CASE WHEN DATEDIFF(DAY, rec.startDate, f.ApptDate) <= 30 THEN rec.startDate ELSE f.ApptDate END
  FROM rec
  JOIN f
    ON rec.rn = f.rn - 1
   AND rec.PatientId = f.PatientId
)
SELECT ApptId, PatientId, ApptDate, Category
FROM rec
ORDER BY PatientId, ApptDate;  

db<>fiddle demo

Output:

+---------+------------+-------------+----------+
| ApptId  | PatientId  |  ApptDate   | Category |
+---------+------------+-------------+----------+
|      1  |       101  | 2020-01-05  | New      |
|      5  |       101  | 2020-01-25  | FollowUp |
|      6  |       101  | 2020-02-12  | New      |
|      7  |       101  | 2020-02-20  | FollowUp |
|      8  |       101  | 2020-03-30  | New      |
|      9  |       303  | 2020-01-28  | New      |
|     10  |       303  | 2020-02-02  | FollowUp |
|      2  |       505  | 2020-01-06  | New      |
|      3  |       505  | 2020-01-10  | FollowUp |
|      4  |       505  | 2020-01-20  | FollowUp |
+---------+------------+-------------+----------+

How it works:

1. f - get starting point(anchor - per every PatientId)
2. rec - recursibe part, if the difference between current value and prev is > 30 change the category and starting point, in context of PatientId
3. Main - display sorted resultset

---

Similar class:

Conditional SUM on Oracle - Capping a windowed function

Session window (Azure Stream Analytics)

Running Total until specific condition is true - Quirky update

---

Addendum

Do not ever use this code on production!

But another option, that is worth mentioning besides using cte, is to use temp table and update in "rounds"

It could be done in "single" round(quirky update):

CREATE TABLE Appt_temp (ApptID INT , PatientID INT, ApptDate DATE, Category NVARCHAR(10))

INSERT INTO Appt_temp(ApptId, PatientId, ApptDate)
SELECT ApptId, PatientId, ApptDate
FROM Appt1;

CREATE CLUSTERED INDEX Idx_appt ON Appt_temp(PatientID, ApptDate);

Query:

DECLARE @PatientId INT = 0,
        @PrevPatientId INT,
        @FirstApptDate DATE = NULL;

UPDATE Appt_temp
SET  @PrevPatientId = @PatientId
    ,@PatientId     = PatientID 
    ,@FirstApptDate = CASE WHEN @PrevPatientId <> @PatientId THEN ApptDate
                           WHEN DATEDIFF(DAY, @FirstApptDate, ApptDate)>30 THEN ApptDate
                           ELSE @FirstApptDate
                      END
    ,Category       = CASE WHEN @PrevPatientId <> @PatientId THEN 'New'
                           WHEN @FirstApptDate = ApptDate THEN 'New'
                           ELSE 'FollowUp' 
                      END
FROM Appt_temp WITH(INDEX(Idx_appt))
OPTION (MAXDOP 1);

SELECT * FROM  Appt_temp ORDER BY PatientId, ApptDate;

db<>fiddle Quirky update

Answer 2

You could do this with a recursive cte. You should first order by apptDate within each patient. That can be accomplished by a run-of-the-mill cte.

Then, in the anchor portion of your recursive cte, select the first ordering for each patient, mark the status as 'new', and also mark the apptDate as the date of the most recent 'new' record.

In the recursive portion of your recursive cte, increment to the next appointment, calculate the difference in days between the present appointment and the most recent 'new' appointment date. If it's greater than 30 days, mark it 'new' and reset the most recent new appointment date. Otherwise mark it as 'follow up' and just pass along the existing days since new appointment date.

Finallly, in the base query, just select the columns you want.

with orderings as (

    select       *, 
                 rn = row_number() over(
                     partition by patientId 
                     order by apptDate
                 ) 
    from         #appt1 a

),

markings as (

    select       apptId, 
                 patientId, 
                 apptDate, 
                 rn, 
                 type = convert(varchar(10),'new'),
                 dateOfNew = apptDate
    from         orderings 
    where        rn = 1

    union all
    select       o.apptId, o.patientId, o.apptDate, o.rn,
                 type = convert(varchar(10),iif(ap.daysSinceNew > 30, 'new', 'follow up')),
                 dateOfNew = iif(ap.daysSinceNew > 30, o.apptDate, m.dateOfNew)
    from         markings m
    join         orderings o 
                     on m.patientId = o.patientId 
                     and m.rn + 1 = o.rn
    cross apply  (select daysSinceNew = datediff(day, m.dateOfNew, o.apptDate)) ap

)

select    apptId, patientId, apptDate, type
from      markings
order by  patientId, rn;

I should mention that I initially deleted this answer because Abhijeet Khandagale's answer seemed to meet your needs with a simpler query (after reworking it a bit). But with your comment to him about your business requirement and your added sample data, I undeleted mine because believe this one meets your needs.

Answer 3

I'm not sure that it's exactly what you implemented. But another option, that is worth mentioning besides using cte, is to use temp table and update in "rounds". So we are going to update temp table while all statuses are not set correctly and build result in an iterative way. We can control number of iteration using simply local variable.

So we split each iteration into two stages.

1. Set all Followup values that are near to New records. That's pretty easy to do just using right filter.
2. For the rest of the records that dont have status set we can select first in group with same PatientID. And say that they are new since they not processed by the first stage.

So

CREATE TABLE #Appt2 (ApptID INT, PatientID INT, ApptDate DATE, AppStatus nvarchar(100))

select * from #Appt1
insert into #Appt2 (ApptID, PatientID, ApptDate, AppStatus)
select a1.ApptID, a1.PatientID, a1.ApptDate, null from #Appt1 a1
declare @limit int = 0;

while (exists(select * from #Appt2 where AppStatus IS NULL) and @limit < 1000)
begin
  set @limit = @limit+1;
  update a2
  set
    a2.AppStatus = IIF(exists(
        select * 
        from #Appt2 a 
        where 
          0 > DATEDIFF(day, a2.ApptDate, a.ApptDate) 
          and DATEDIFF(day, a2.ApptDate, a.ApptDate) > -30 
          and a.ApptID != a2.ApptID 
          and a.PatientID = a2.PatientID
          and a.AppStatus = 'New'
          ), 'Followup', a2.AppStatus)
  from #Appt2 a2

  --select * from #Appt2

  update a2
  set a2.AppStatus = 'New'
  from #Appt2 a2 join (select a.*, ROW_NUMBER() over (Partition By PatientId order by ApptId) rn from (select * from #Appt2 where AppStatus IS NULL) a) ar
  on a2.ApptID = ar.ApptID
  and ar.rn = 1

  --select * from #Appt2

end

select * from #Appt2 order by PatientID, ApptDate

drop table #Appt1
drop table #Appt2

Update. Read the comment provided by Lukasz. It's by far smarter way. I leave my answer just as an idea.

Answer 4

I believe the recursive common expression is great way to optimize queries avoiding loops, but in some cases it can lead to bad performance and should be avoided if possible.

I use the code below to solve the issue and test it will more values, but encourage you to test it with your real data, too.

WITH DataSource AS
(
    SELECT *
          ,CEILING(DATEDIFF(DAY, MIN([ApptDate]) OVER (PARTITION BY [PatientID]), [ApptDate]) * 1.0 / 30 + 0.000001) AS [GroupID]
    FROM #Appt1
)
SELECT *
     ,IIF(ROW_NUMBER() OVER (PARTITION BY [PatientID], [GroupID] ORDER BY [ApptDate]) = 1, 'New', 'Followup')
FROM DataSource
ORDER BY [PatientID]
        ,[ApptDate];

The idea is pretty simple - I want separate the records in group (30 days), in which group the smallest record is new, the others are follow ups. Check how the statement is built:

SELECT *
      ,DATEDIFF(DAY, MIN([ApptDate]) OVER (PARTITION BY [PatientID]), [ApptDate])
      ,DATEDIFF(DAY, MIN([ApptDate]) OVER (PARTITION BY [PatientID]), [ApptDate]) * 1.0 / 30
      ,CEILING(DATEDIFF(DAY, MIN([ApptDate]) OVER (PARTITION BY [PatientID]), [ApptDate]) * 1.0 / 30 + 0.000001) 
FROM #Appt1
ORDER BY [PatientID]
        ,[ApptDate];

So:

1. first, we are getting the first date, for each group and calculating the differences in days with the current one
2. then, we are want to get groups - * 1.0 / 30 is added
3. as for 30, 60, 90, etc days we are getting whole number and we wanted to start a new period, I have added + 0.000001; also, we are using ceiling function to get the smallest integer greater than, or equal to, the specified numeric expression

That's it. Having such group we simply use ROW_NUMBER to find our start date and make it as new and leaving the rest as follow ups.

Answer 5

Although it's not clearly addressed in the question, it's easy to figure out that the appointment dates cannot be simply categorized by 30-day groups. It makes no business sense. And you cannot use the appt id either. One can make a new appointment today for 2020-09-06. Here is how I address this issue. First, get the first appointment, then calculate the date difference between each appointment and the first appt. If it's 0, set to 'New'. If <= 30 'Followup'. If > 30, set as 'Undecided' and do the next round check until there is no more 'Undecided'. And for that, you really need a while loop, but it does not loop through each appointment date, rather only a few datasets. I checked the execution plan. Even though there are only 10 rows, the query cost is significantly lower than that using recursive CTE, but not as low as Lukasz Szozda's addendum method.

IF OBJECT_ID('tempdb..#TEMPTABLE') IS NOT NULL DROP TABLE #TEMPTABLE
SELECT ApptID, PatientID, ApptDate
    ,CASE WHEN (DATEDIFF(DAY, MIN(ApptDate) OVER (PARTITION BY PatientID), ApptDate) = 0) THEN 'New' 
    WHEN (DATEDIFF(DAY, MIN(ApptDate) OVER (PARTITION BY PatientID), ApptDate) <= 30) THEN 'Followup'
    ELSE 'Undecided' END AS Category
INTO #TEMPTABLE
FROM #Appt1

WHILE EXISTS(SELECT TOP 1 * FROM #TEMPTABLE WHERE Category = 'Undecided') BEGIN
    ;WITH CTE AS (
        SELECT ApptID, PatientID, ApptDate 
            ,CASE WHEN (DATEDIFF(DAY, MIN(ApptDate) OVER (PARTITION BY PatientID), ApptDate) = 0) THEN 'New' 
            WHEN (DATEDIFF(DAY, MIN(ApptDate) OVER (PARTITION BY PatientID), ApptDate) <= 30) THEN 'Followup'
            ELSE 'Undecided' END AS Category    
        FROM #TEMPTABLE
        WHERE Category = 'Undecided'
    )
    UPDATE #TEMPTABLE
    SET Category = CTE.Category
    FROM #TEMPTABLE t
        LEFT JOIN CTE ON CTE.ApptID = t.ApptID
    WHERE t.Category = 'Undecided'
END

SELECT ApptID, PatientID, ApptDate, Category 
FROM #TEMPTABLE

Answer 6

With due respect to everybody and in IMHO,

There is not much difference between While LOOP and Recursive CTE in terms of RBAR

There is not much performance gain when using Recursive CTE and Window Partition function all in one.

Appid should be int identity(1,1) , or it should be ever increasing clustered index.

Apart from other benefit it also ensure that all successive row APPDate of that patient must be greater.

This way you can easily play with APPID in your query which will be more efficient than putting inequality operator like >,< in APPDate. Putting inequality operator like >,< in APPID will aid Sql Optimizer.

Also there should be two date column in table like

APPDateTime datetime2(0) not null,
Appdate date not null

As these are most important columns in most important table,so not much cast ,convert.

So Non clustered index can be created on Appdate

Create NonClustered index ix_PID_AppDate_App  on APP (patientid,APPDate) include(other column which is not i predicate except APPID)

Test my script with other sample data and lemme know for which sample data it not working. Even if it do not work then I am sure it can be fix in my script logic itself.

CREATE TABLE #Appt1 (ApptID INT, PatientID INT, ApptDate DATE)
INSERT INTO #Appt1
SELECT  1,101,'2020-01-05'  UNION ALL
SELECT  2,505,'2020-01-06'  UNION ALL
SELECT  3,505,'2020-01-10'  UNION ALL
SELECT  4,505,'2020-01-20'  UNION ALL
SELECT  5,101,'2020-01-25'  UNION ALL
SELECT  6,101,'2020-02-12'  UNION ALL
SELECT  7,101,'2020-02-20'  UNION ALL
SELECT  8,101,'2020-03-30'  UNION ALL
SELECT  9,303,'2020-01-28'  UNION ALL
SELECT  10,303,'2020-02-02' 

;With CTE as
(
select a1.* ,a2.ApptDate as NewApptDate
from #Appt1 a1
outer apply(select top 1 a2.ApptID ,a2.ApptDate
from #Appt1 A2 
where a1.PatientID=a2.PatientID and a1.ApptID>a2.ApptID 
and DATEDIFF(day,a2.ApptDate, a1.ApptDate)>30
order by a2.ApptID desc )A2
)
,CTE1 as
(
select a1.*, a2.ApptDate as FollowApptDate
from CTE A1
outer apply(select top 1 a2.ApptID ,a2.ApptDate
from #Appt1 A2 
where a1.PatientID=a2.PatientID and a1.ApptID>a2.ApptID 
and DATEDIFF(day,a2.ApptDate, a1.ApptDate)<=30
order by a2.ApptID desc )A2
)
select  * 
,case when FollowApptDate is null then 'New' 
when NewApptDate is not null and FollowApptDate is not null 
and DATEDIFF(day,NewApptDate, FollowApptDate)<=30 then 'New'
else 'Followup' end
 as Category
from cte1 a1
order by a1.PatientID

drop table #Appt1

Answer 7

I hope this will help you.

WITH CTE AS
(
    SELECT #Appt1.*, RowNum = ROW_NUMBER() OVER (PARTITION BY PatientID ORDER BY ApptDate, ApptID) FROM #Appt1
)

SELECT A.ApptID , A.PatientID , A.ApptDate ,
Expected_Category = CASE WHEN (DATEDIFF(MONTH, B.ApptDate, A.ApptDate) > 0) THEN 'New' 
WHEN (DATEDIFF(DAY, B.ApptDate, A.ApptDate) <= 30) then 'Followup' 
ELSE 'New' END
FROM CTE A
LEFT OUTER JOIN CTE B on A.PatientID = B.PatientID 
AND A.rownum = B.rownum + 1
ORDER BY A.PatientID, A.ApptDate

Answer 8

You could use a Case statement.

select 
      *, 
      CASE 
          WHEN DATEDIFF(d,A1.ApptDate,A2.ApptDate)>30 THEN 'New' 
          ELSE 'FollowUp' 
      END 'Category'
from 
      (SELECT PatientId, MIN(ApptId) 'ApptId', MIN(ApptDate) 'ApptDate' FROM #Appt1 GROUP BY PatientID)  A1, 
      #Appt1 A2 
where 
     A1.PatientID=A2.PatientID AND A1.ApptID<A2.ApptID

The question is, should this category be assigned based off the initial appointment, or the one prior? That is, if a Patient has had three appointments, should we compare the third appointment to the first, or the second?

You problem states the first, which is how I've answered. If that's not the case, you'll want to use lag.

Also, keep in mind that DateDiff makes not exception for weekends. If this should be weekdays only, you'll need to create your own Scalar-Valued function.

Answer 9

using Lag function

select  apptID, PatientID , Apptdate ,  
    case when date_diff IS NULL THEN 'NEW' 
         when date_diff < 30 and (date_diff_2 IS NULL or date_diff_2 < 30) THEN  'Follow Up'
         ELSE 'NEW'
    END AS STATUS FROM 
(
select 
apptID, PatientID , Apptdate , 
DATEDIFF (day,lag(Apptdate) over (PARTITION BY PatientID order by ApptID asc),Apptdate) date_diff ,
DATEDIFF(day,lag(Apptdate,2) over (PARTITION BY PatientID order by ApptID asc),Apptdate) date_diff_2
  from #Appt1
) SRC

Demo --> https://rextester.com/TNW43808

Answer 10

with cte
as
(
select 
tmp.*, 
IsNull(Lag(ApptDate) Over (partition by PatientID Order by  PatientID,ApptDate),ApptDate) PriorApptDate
 from #Appt1 tmp
)
select 
PatientID, 
ApptDate, 
PriorApptDate, 
DateDiff(d,PriorApptDate,ApptDate) Elapsed,
Case when DateDiff(d,PriorApptDate,ApptDate)>30 
or DateDiff(d,PriorApptDate,ApptDate)=0 then 'New' else 'Followup'   end Category   from cte

Mine is correct. The authors was incorrect, see elapsed