0

I got a performance issue with an import task (user need to import new data into the system) and could use little help.

The data within the database looks like this:

Table Categories:

Columns:

  • Id int primary key identity(1,1)
  • Code nvarchar(20) not null
  • RootId int not null
  • ParentId int not null

Table CategoryNames:

Columns:

  • CategoryId int not null foreign key references Category (Id)
  • LanguageId int not null foreign key references Language (Id)
  • Name nvarchar(100)

Currently it's working like this: for each row

  • Create connection to database
  • Call stored procedure with data[i]
  • Close connection

I already got rid of the creating and closing connection for each row of data. But it's still not good enough.

Currently the task needs round about 36 minutes to insert ~22000 categories. The stored procedure looks like this:

procedure Category_Insert 
     (@code nvarchar(20),
      @root int,
      @parent int,
      @languageId int,
      @name nvarchar(100),
      @id int output)
as
     set nocount on

     insert into Categories (Code, RootId, ParentId) 
     values (@code, @root, @parent)

     select id = scope_identity()

     insert into CategoryNames (CategoryId, LanguageId, Name) 
     values (@id, @languageId, @name)

Got any advice how I can speed up the performance of that task?

I would love to use bulk insert or something like that, but how would I realize the logic of the stored procedure with bulk insert?

Or is the any other way to speed this up?

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
  • 1
    Possible duplicate of [Fastest way to perform nested bulk inserts with scope\_identity() usage?](http://stackoverflow.com/questions/34826450/fastest-way-to-perform-nested-bulk-inserts-with-scope-identity-usage) – Zohar Peled Jan 12 '17 at 13:41
  • This is inserting each row one at a time, right? Can you not import the data all at once? – iamdave Jan 12 '17 at 15:25
  • Is this a one time only update or something that is to be done multiple times on a periodic basis? – JohnH Jan 12 '17 at 15:49
  • @JohnH: This task is executed multiple times, I've been told several times a week. – Marek Cwolek Jan 12 '17 at 16:09
  • @iamdave: Yes correct, each row is inserted one at a time. Trying to speed it up without rewriting the whole task. – Marek Cwolek Jan 12 '17 at 16:13
  • @ZoharPeled: Thanks for the link, I'll check it out. Maybe that's exactly what I need. – Marek Cwolek Jan 12 '17 at 16:13
  • You seem to need an ETL (Extract Transfer and Load) program. Have you searched SO using the "etl" key word? – JohnH Jan 12 '17 at 18:03

0 Answers0