0

I have a requirement to insert/update bulk number of records into a table. For that, i have written a stored procedure as follows. But it is taking much time to execute. Could any one suggest changes to SP to get better performance in execution.

create  procedure sp_save_user
(
@a_i_lang_id        integer,
@a_s_data           ntext
)
WITH ENCRYPTION
as
begin   
set nocount on
SET QUOTED_IDENTIFIER ON 
--Declaring local variables
declare @l_s_USER_ID NVARCHAR(30)
declare @l_s_USER_NAME NVARCHAR(255)

declare @l_n_rmStatusCount numeric(10)
declare @l_n_XMLDoc XML

set @l_n_XMLDoc = cast(@a_s_data as xml)

CREATE TABLE #DW_TEMP_TABLE_SAVE(  
[USER_ID] [NVARCHAR](30), 
[USER_NAME] [NVARCHAR](255)
)   
insert into #DW_TEMP_TABLE_SAVE
select A.B.value('(USER_ID)[1]', 'nvarchar(30)' ) [USER_ID], 
A.B.value('(USER_NAME)[1]', 'nvarchar(30)' ) [USER_NAME]
from @l_n_XMLDoc.nodes('//ROW') as A(B) 

--Get total number od records
select @l_n_rmStatusCount = count(*) from #DW_TEMP_TABLE_SAVE

--loop through records and insert/update table
while (@l_n_rmStatusCount > 0)
begin
    SELECT  @l_s_USER_ID        =   [USER_ID] , 
            @l_s_USER_NAME          =   [USER_NAME]         
    FROM ( SELECT   ROW_NUMBER() OVER (ORDER BY [USER_ID]) AS rownumber,
        [USER_ID],[USER_NAME]  FROM #DW_TEMP_TABLE_SAVE) as temptablename
    WHERE rownumber = @l_n_rmStatusCount
    if exists(
    select 'X' from table_user_info(nolock)
    where [user_id]                 = @l_s_USER_ID      
    )
    begin
        -- do update 
    end
    else
    begin
        -- do insert 
    end     
    set @l_n_rmStatusCount = @l_n_rmStatusCount -1
end
drop table #DW_TEMP_TABLE_SAVE   
SET QUOTED_IDENTIFIER OFF       
set nocount off
end
go
user1587872
  • 345
  • 1
  • 3
  • 14

2 Answers2

1

Never EVER perform large amounts of INSERTS/UPDATES using a WHILE loop!!!

Use the T-SQL MERGE statement when you want to insert some records while updating other records at the same time. Take a look at the examples or come back here and ask for more specific questions if you have trouble getting it to work.

Dan
  • 10,480
  • 23
  • 49
  • Although the MERGE is a nice way to do things, I've noticed that it gets quite tricky when you do updates without affecting the other records that didn't change. – user3036342 Apr 09 '14 at 10:51
  • It can get tricky depending on the table. For example, if we have 4 states, all of which has data in 1 table, and we get an update/insert/delete from 1 state only, it gets tricky because it wipes the other 3 states since you didn't give it a full dataset to compare with. So the merge deletes the other records – user3036342 Apr 09 '14 at 10:54
  • I think you're misunderstanding a few concepts about MERGE. MERGE lets you specify in great detail what should happen when records exist in the destination table but no longer exist in the source table and vice versa. You can easily create a MERGE statement that doesn't delete any records. – Dan Apr 09 '14 at 10:55
  • The problem comes in when you do have to delete records :) the source can have a few records deleted. When using merge, it then synchronizes the data. As I pointed out, if you only send through 1 state (in my example, like Texas), the merge will see it as "all other records should now be deleted because the other 3 states are missing". Like I said. It can get tricky. Because now you HAVE to supply a full recordset each time to do a merge. Whereas with the methods I described, you can do partial updates, inserts and deletes – user3036342 Apr 09 '14 at 10:58
  • That's not true. In the `WHEN NOT MATCHED BY SOURCE` part of the MERGE-statement (which is the one you typically use to delete records), you may optionally specify a condition, such as `WHEN NOT MATCHED BY SOURCE AND State = 'Texas'`, to limit the MERGE operation to only parts of your data. – Dan Apr 09 '14 at 11:04
  • @Dan: Thanks for that. I tried to use MERGE, but it is expecting only INSERT/UPDATE/DELETE. Sorry if i misleaded you with the SP i posted above. Actually it won't be direct insert/update records into target table. I have to call again some other SPs there based on the record existance. Please help me out. – user1587872 Apr 09 '14 at 11:24
  • We tried what you mentioned too with not much success. Can't hardcode for every state and using a variable gives us the same outcome. As soon as only a specific state's records comes through, all others are deleted. We did go a while withou the delete statement, until we settled on the best solution as I mentioned in my answer – user3036342 Apr 09 '14 at 11:27
  • Hmm okay - it is my experience that MERGE can be used successfully in most cases. If you like, I'd be happy to take a look at your particular situation. I just need some more details on what you are currently doing - just PM me in case you would like me to take a look. – Dan Apr 09 '14 at 11:34
  • @Dan:PM means? (..just PM me... ) – user1587872 Apr 09 '14 at 11:39
  • Ahh - I'm such a StackOverflow noob... apparently, Private Messages (PM) is not possible here. Well then - if you feel up to it, feel free to start a new question regarding how to replace your existing SQL with MERGE, and post the link to the question here as a comment - then I will be happy to take a look. – Dan Apr 09 '14 at 11:54
  • @Dan: http://stackoverflow.com/questions/22962726/xml-handling-with-bulk-records-in-sql – user1587872 Apr 09 '14 at 12:57
  • @Dan: Problem is solved for my case. http://stackoverflow.com/questions/22978955/alternatives-to-xml-shredding-in-sql/22979219?noredirect=1#comment35092033_22979219 – user1587872 Apr 10 '14 at 06:26
1

A few things. Get rid of the cursor. Use table variables instead of #temp types, bulk update/insert. Use xml data type out of the gate instead of converting it later on in the code

For example.

Input variables:

create  procedure sp_save_user
(
@a_i_lang_id        integer,
@a_s_data           xml
)

Table variable:

DECLARE @DW_TEMP_TABLE_SAVE AS TABLE (  
[USER_ID] [NVARCHAR](30), 
[USER_NAME] [NVARCHAR](255)
)  

Bulk insert:

INSERT INTO your_table (Column1,Column2)
select Column1,Column2 from @DW_TEMP_TABLE_SAVE where Column1 not in (select column1 from your_table) --Provided Column1 is the key, otherwise just do a left join and get records where the left joined table is null

Bulk update:

Update destination
set destination.Column1 = source.Column1, destination.Column2 = source.Column2
from your_table destination
inner join @DW_TEMP_TABLE_SAVE source on destination.Column1 = source.Column1 and destination.Column2 = source.Column2

The above methods show you how to insert records if they don't exist, update records that do exist from the source table (which is the changes), all without a cursor, which is quite heavy in processing

Hope this helps

user3036342
  • 1,023
  • 7
  • 15
  • FYI, do the update first, then the insert. Otherwise you will update all the inserted records and cause unnecessary processing – user3036342 Apr 09 '14 at 10:52