2

I am looking for a way to pull XML from a SQL database and shred the XML via SSIS in bulk. I currently have a package that can pull XML from the database and pass the XML to a stored procedure, via variable, for shredding but this only works 1 record at a time. When processing 100,000 records, this can become quite time consuming.

I would like to shred multiple XML values at once using SSIS. Is this possible with SSIS? Perhaps something in a Data Flow Task where all the XML values are selected from a source then passed to a parser of some sort?

Chris
  • 838
  • 1
  • 15
  • 27

1 Answers1

2

So after a lot of searching and thinking about the problem, I redesigned the stored procedures that did the shredding. Instead of shredding a variable being passed into the stored procedure, I just shredded from the XML column itself. This allowed me to shred many XML values at once instead of looping and passing them to a stored procedure one at a time. This gave me the performance boost I was looking for. So instead of something like this...

SET @xmlData = CAST(@xmlMessageData AS XML)

SELECT
   , row.value('xmlNode1[1]', 'BIT' )                   AS Column1  
   , row.value('xmlNode2[1]', 'NVARCHAR(255)')          AS Column2
   , row.value('xmlNode3[1]', 'BIT' )                   AS Column3
   , row.value('xmlNode4[1]/Name[1]', 'NVARCHAR(255)' ) AS Column4
FROM @xmlData.nodes('xmlRootNode') AS T1(row)

I now do this...

SELECT
   , row.value('xmlNode1[1]', 'BIT' )                   AS Column1  
   , row.value('xmlNode2[1]', 'NVARCHAR(255)')          AS Column2
   , row.value('xmlNode3[1]', 'BIT' )                   AS Column3
   , row.value('xmlNode4[1]/Name[1]', 'NVARCHAR(255)' ) AS Column4
FROM [ESBMessagesData] D
CROSS APPLY
    [DataItem].nodes('xmlRootNode') AS T1(row)
Chris
  • 838
  • 1
  • 15
  • 27