208

How do I loop through a set of records from a select statement?

Say I have a few records that I wish to loop through and do something with each record. Here's a primitive version of my select statement:

select top 1000 * from dbo.table
where StatusID = 7
Himanshu
  • 31,810
  • 31
  • 111
  • 133
Funky
  • 12,890
  • 35
  • 106
  • 161
  • 5
    What do you want to do to each record? The preference would be to do the work in a SQL query. Barring that you would need to use T-SQL, perhaps with cursors. – Gordon Linoff Dec 18 '13 at 15:42
  • 2
    I would use a Cursor. – FloChanz Dec 18 '13 at 15:42
  • I want to call another stored proc for each record – Funky Dec 18 '13 at 15:43
  • 5
    That will be quite slow - is it not possible to re-write the stored proc or move some of the logic out of it to work in a set-based manner? – Bridge Dec 18 '13 at 15:45
  • 2
    @Funky what does the sproc do? Often code can be re-written in a set based manner (i.e. avoid loops). If you're adamant you want to perform an RBAR operation (https://www.simple-talk.com/sql/t-sql-programming/rbar--row-by-agonizing-row/) then a cursor is the thing you want to investigate. – gvee Dec 18 '13 at 15:50
  • 1
    Perhaps you can explain what you will be doing with this data in more detail. In most cases you can easily write a single SQL query that will do what you need to get done in one action instead of looping through individual records. – Alan Barber Dec 18 '13 at 15:52
  • The requirement is to loop through the data which calls a sp which then send an email. Hey, I didn't design the system! – Funky Dec 18 '13 at 15:59
  • 1
    I would recommend taking the loop out of the database entirely. Query the data from an app/script, loop through the resultset and send your emails there. – alroc Dec 18 '13 at 16:32
  • @GordonLinoff Cursors can be a bad choice to do looping, as its slow while looping through a large number of records. – Biswa Aug 03 '18 at 14:16
  • 1
    Here is a nice solution - https://www.coderjony.com/blogs/how-to-loop-through-a-table-variable-in-sql-server/ – Ankush Jain Feb 07 '20 at 07:02

8 Answers8

279

By using T-SQL and cursors like this :

DECLARE @MyCursor CURSOR;
DECLARE @MyField YourFieldDataType;
BEGIN
    SET @MyCursor = CURSOR FOR
    select top 1000 YourField from dbo.table
        where StatusID = 7      

    OPEN @MyCursor 
    FETCH NEXT FROM @MyCursor 
    INTO @MyField

    WHILE @@FETCH_STATUS = 0
    BEGIN
      /*
         YOUR ALGORITHM GOES HERE   
      */
      FETCH NEXT FROM @MyCursor 
      INTO @MyField 
    END; 

    CLOSE @MyCursor ;
    DEALLOCATE @MyCursor;
END;
Tejasvi Hegde
  • 2,694
  • 28
  • 20
FloChanz
  • 3,279
  • 1
  • 15
  • 15
  • 5
    The right thing is to rewrite teh process so that it does not need to loop. Looping is an extremely bad choice in a database. – HLGEM Dec 18 '13 at 15:54
  • 36
    Perhaps you're right but with the information given in the question at the time I wrote the answer the user just want to loop through a set of data...and a Cursor is a way to do it. – FloChanz Dec 18 '13 at 16:02
  • 25
    Cursors are just a tool -- nothing generally right or wrong about them. Observe the performance and decide. This answer (cursors) is one possible choice. You can also use a WHILE LOOP, CTE, etc. – Chains Dec 18 '13 at 16:09
  • 1
    Can you fetch more than one variable ? Let's say that I need result from three columns. Can I get those from cursor ? – FrenkyB Jul 08 '15 at 11:07
  • 2
    @FrenkyB Yes you can. Look this way... http://stackoverflow.com/questions/11035187/how-do-i-fetch-multiple-columns-for-use-in-a-cursor-loop – sam yi Jul 14 '15 at 15:55
  • 1
    What is the purpose of the outer `BEGIN` and `END` there? I tried the code with and without and I get the same results. – JamesFaix Jun 17 '16 at 13:21
  • 2
    Congratulations, your solution is even on msdn: https://msdn.microsoft.com/en-us/library/ms180152.aspx#B-Using-FETCH-to-store-values-in-variables and I really like how you use the Field Data Type. – Pete Sep 15 '16 at 22:47
  • What do you mean looping over records without using cursors, @Chains ? What are CTEs? – bitoolean Apr 17 '18 at 20:57
  • @HLGEM In stored procedures you sometimes need cursors, when you perform actions such as running other procedures for each record. I now needed to reset the seed for all tables with an IDENTITY set in a database and I don't see another way of doing it than using cursors. – bitoolean Apr 17 '18 at 21:02
  • Why does the first FETCH try to get the NEXT? Wouldn't it read better if it was a FETCH FIRST? I get why it says FETCH NEXT inside the loop... – bitoolean Apr 17 '18 at 21:06
  • @bitoolean CTE is 'Common Table Expression' in TSQL, supports recursion. Tons of material on that on in SO, Microsoft documentation, Google generally... Lots of handy uses for CTEs. – Chains Apr 26 '18 at 15:02
  • 1
    @FloChanz You may want to update your cursor to use "LOCAL FAST_FORWARD" as that is usually WAY more performant: DECLARE @ MyCursor CURSOR LOCAL FAST_FORWARD; – DBADon Oct 16 '20 at 14:42
  • @bitoolean FIRST is not allowed in this case - you get a "fetch type first cannot be used with forward only cursors"-error. – Stefan Jelkovich May 16 '22 at 18:07
  • Cursors are deprecated, you have to use 'while loop' or filter rows using temp tables, or combine both of this solutions (refer to next answers) – giorgio calarco Oct 12 '22 at 09:42
  • @giorgiocalarco Cursors are not deprecated. What are you talking about? – TylerH Oct 28 '22 at 14:53
  • @TylerH https://stackoverflow.com/questions/58141/why-is-it-considered-bad-practice-to-use-cursors-in-sql-server cursors cause very often many problems on performances, more than benefits – giorgio calarco Feb 28 '23 at 11:12
  • @giorgiocalarco "often causes problems" is not the same thing as "deprecated". Please don't use words that are not correct. Cursors are easy to use correctly and are sometimes the only solution. Please do not spread misinformation! – TylerH Feb 28 '23 at 15:55
  • Sorry for the misword, I should say strongly "not-recommended" instead of "deprecated", but I disagree with you that it's misinformation, because as documented cursors have serious performance problems and are not the only solution. – giorgio calarco Feb 28 '23 at 16:19
150

This is what I've been doing if you need to do something iterative... but it would be wise to look for set operations first. Also, do not do this because you don't want to learn cursors.

select top 1000 TableID
into #ControlTable 
from dbo.table
where StatusID = 7

declare @TableID int

while exists (select * from #ControlTable)
begin

    select top 1 @TableID = TableID
    from #ControlTable
    order by TableID asc

    -- Do something with your TableID

    delete #ControlTable
    where TableID = @TableID

end

drop table #ControlTable
sam yi
  • 4,806
  • 1
  • 29
  • 40
  • Thank you! My Example with update and group by logic using above code : http://pastebin.com/GAjUNNi9. Maybe will be useful to anybody. – Nigrimmist Jan 11 '16 at 15:27
  • can the variable be used as a column name in update statement inside the loop? Something like "Update TableName SET @ColumnName=2" – M H Jan 20 '16 at 16:21
  • 3
    This actually answers the original question, as it lets you iterate over the entire row; whereas cursor lets you iterate over a particular column in a row. – I_do_python Apr 26 '18 at 19:50
  • 2
    `Cursors` are more verbose than `while` statements, but they're more optimized. I wouldn't recommend a `while` statement, especially with a subquery in the loop condition. – Christiano Kiss Apr 16 '19 at 03:50
45

Small change to sam yi's answer (for better readability):

select top 1000 TableID
into #ControlTable 
from dbo.table
where StatusID = 7

declare @TableID int

while exists (select * from #ControlTable)
begin

    select @TableID = (select top 1 TableID
                       from #ControlTable
                       order by TableID asc)

    -- Do something with your TableID

    delete #ControlTable
    where TableID = @TableID

end

drop table #ControlTable
Cody Gray - on strike
  • 239,200
  • 50
  • 490
  • 574
Precept
  • 547
  • 4
  • 2
26

By using cursor you can easily iterate through records individually and print records separately or as a single message including all the records.

DECLARE @CustomerID as INT;
declare @msg varchar(max)
DECLARE @BusinessCursor as CURSOR;

SET @BusinessCursor = CURSOR FOR
SELECT CustomerID FROM Customer WHERE CustomerID IN ('3908745','3911122','3911128','3911421')

OPEN @BusinessCursor;
    FETCH NEXT FROM @BusinessCursor INTO @CustomerID;
    WHILE @@FETCH_STATUS = 0
        BEGIN
            SET @msg = '{
              "CustomerID": "'+CONVERT(varchar(10), @CustomerID)+'",
              "Customer": {
                "LastName": "LastName-'+CONVERT(varchar(10), @CustomerID) +'",
                "FirstName": "FirstName-'+CONVERT(varchar(10), @CustomerID)+'",    
              }
            }|'
        print @msg
    FETCH NEXT FROM @BusinessCursor INTO @CustomerID;
END
Agnel Amodia
  • 765
  • 8
  • 18
  • 1
    this looks interesting. I wonder what the @ identifier means. – netskink Oct 24 '19 at 14:21
  • 1
    @ is just to differentiate as variables. – Agnel Amodia Nov 13 '19 at 15:44
  • 2
    This is definitely interesting. But please enlighten me: ---> How the 'SET @msg ...' section getting the Data from Customer? And how are the Field identified as LastName or FirstName returning the value? From which variable? I don't understand that part if you may explain please – Tsiry Rakotonirina Mar 14 '21 at 11:32
9

Just another approach if you are fine using temp tables.I have personally tested this and it will not cause any exception (even if temp table does not have any data.)

CREATE TABLE #TempTable
(
    ROWID int identity(1,1) primary key,
    HIERARCHY_ID_TO_UPDATE int,
)

--create some testing data
--INSERT INTO #TempTable VALUES(1)
--INSERT INTO #TempTable VALUES(2)
--INSERT INTO #TempTable VALUES(4)
--INSERT INTO #TempTable VALUES(6)
--INSERT INTO #TempTable VALUES(8)

DECLARE @MAXID INT, @Counter INT

SET @COUNTER = 1
SELECT @MAXID = COUNT(*) FROM #TempTable

WHILE (@COUNTER <= @MAXID)
BEGIN
    --DO THE PROCESSING HERE 
    SELECT @HIERARCHY_ID_TO_UPDATE = PT.HIERARCHY_ID_TO_UPDATE
    FROM #TempTable AS PT
    WHERE ROWID = @COUNTER

    SET @COUNTER = @COUNTER + 1
END


IF (OBJECT_ID('tempdb..#TempTable') IS NOT NULL)
BEGIN
    DROP TABLE #TempTable
END
David Ferenczy Rogožan
  • 23,966
  • 9
  • 79
  • 68
Sandeep
  • 615
  • 6
  • 13
  • This is really weird. It contains a lot of errors, also using of two variables where one goes from 1 to `COUNT(*)` and second goes from `COUNT(*)` to 1 is weird. – David Ferenczy Rogožan Feb 09 '16 at 16:50
  • The variable MAXID is used to LOOP through. The variable COUNTER is used to perform an operation on a particular record in the table. If I read the question it talks about " have a few records that I wish to loop through and do something with each record". I may be wrong but please point out what is wrong above @DAWID – Sandeep Feb 09 '16 at 22:22
  • 2
    I think it's obvious how you use those variables in your code. You can just have `WHILE (@COUTNER <= @ROWID)` and you don't need to decrement `@ROWID` in each iteration. BTW what happens if `ROWID`s in your table are not continuous (some rows were previously deleted). – David Ferenczy Rogožan Feb 10 '16 at 10:24
  • 1
    When would you suggest using a Temp Table over using a Cursor? Is this merely a design choice, or does one have better performance? – h0r53 Jun 16 '16 at 19:22
8

You could choose to rank your data and add a ROW_NUMBER and count down to zero while iterate your dataset.

-- Get your dataset and rank your dataset by adding a new row_number
SELECT  TOP 1000 A.*, ROW_NUMBER() OVER(ORDER BY A.ID DESC) AS ROW
INTO #TEMPTABLE 
FROM DBO.TABLE AS A
WHERE STATUSID = 7;

--Find the highest number to start with
DECLARE @COUNTER INT = (SELECT MAX(ROW) FROM #TEMPTABLE);
DECLARE @ROW INT;

-- Loop true your data until you hit 0
WHILE (@COUNTER != 0)
BEGIN

    SELECT @ROW = ROW
    FROM #TEMPTABLE
    WHERE ROW = @COUNTER
    ORDER BY ROW DESC

    --DO SOMTHING COOL  

    -- SET your counter to -1
    SET @COUNTER = @ROW -1
END

DROP TABLE #TEMPTABLE
Bunkerbuster
  • 963
  • 9
  • 17
2

this way we can iterate into table data.

DECLARE @_MinJobID INT
DECLARE @_MaxJobID INT
CREATE  TABLE #Temp (JobID INT)

INSERT INTO #Temp SELECT * FROM DBO.STRINGTOTABLE(@JobID,',')
SELECT @_MinJID = MIN(JobID),@_MaxJID = MAX(JobID)  FROM #Temp

    WHILE @_MinJID <= @_MaxJID
    BEGIN

        INSERT INTO Mytable        
        (        
            JobID,        
        )        

        VALUES        
        (        
            @_MinJobID,        
        ) 

        SET @_MinJID = @_MinJID + 1;
    END

DROP TABLE #Temp

STRINGTOTABLE is user define function which will parse comma separated data and return table. thanks

Monojit Sarkar
  • 2,353
  • 8
  • 43
  • 94
0

I think this is the easy way example to iterate item.

declare @cateid int
select CateID into [#TempTable] from Category where GroupID = 'STOCKLIST'

while (select count(*) from #TempTable) > 0
begin
    select top 1 @cateid = CateID from #TempTable
    print(@cateid)

    --DO SOMETHING HERE

    delete #TempTable where CateID = @cateid
end

drop table #TempTable
江明哲
  • 27
  • 5