Split A Single Field Value Into Multiple Fixed-Length Column Values in T-SQL

Question

I've looked at about 15 different answers on SO but haven't found this exact situation yet. I'm doing a custom data export and need to export to a data file that will be imported into an older system that needs the data in a specific length/format. I have a "MEMO" column that can have a large amount of text in it. I need to select that value and split it into multiple columns with a FIXED length of 75 chars. For instance, if I have a row with a message that is 185 chars, I need to split that into 3 new columns of 75 chars, MEMO1, MEMO2, MEMO3, with the remaining space in MEMO3 being filled with spaces to equal the 75 chars. The other catch, I can only use up to 18 75-char columns to dump the data into. If it's longer than 1350 (18x75) chars, the rest gets truncated.

I tried this approach, but it doesn't take the total number of new memo columns into consideration. I need some way to iterate over NUMBEROFMEMOS and only select the necessary amount of new MEMO columns, but apparently you can't do a WHILE in a select.

SELECT FIRSTNAME, 
       LASTNAME, 
       DOB, 
       CEILING(LEN(NOTETEXT) / 75.0) as NUMBEROFMEMOS,
       SUBSTRING(NOTETEXT, 1, 75) as MEMOLINE1,
       SUBSTRING(NOTETEXT, 76, 149) as MEMOLINE2,
       SUBSTRING(NOTETEXT, 150, 224) as MEMOLINE3,
       etc. etc. etc
FROM CUSTOMER

I'm a long-time application dev who is trying to get more involved in the DB side of things. If I were in the C# world, I would just create a method to do a for loop up to NUMBEROFMEMOS and output the data that way. I don't think that works here though. Thanks in advance!

If you're thinking of writing a query that doesn't have a pre-defined number of columns, the only way to do that is with dynamic SQL. — Tab Alleman, Jun 28 '16 at 14:36
not related to the problem by try use `_` to separate words or lower/uppercase in your variables to make it easy to read. `NOTETEXT` could be `NOTE_TEXT` or `noteText`. I swear was trying to figure out what are `NoTetext` — Juan Carlos Oropeza, Jun 28 '16 at 14:49
you could write a stored procedure where you can loop through your records similar as in c#. There you can create a temp table with enough 75 length columns and fill that in the loop and at the end just return that temp table — GuidoG, Jun 28 '16 at 14:57

score 3 · Answer 1 · edited May 23 '17 at 12:33

As you are .net developer I guess it will be easy for you to write a .net function that you can use in your T-SQL code. In order to write SQL CLR functions check this answer (I have used one of the links to implement SQL CLR regex function.

Let's say you need to split the values by 4-length chunks and show maximum 6 of them:

DECLARE @DataSouce TABLE
(
    [RecordID] TINYINT IDENTITY(1,1) PRIMARY KEY
   ,[RecordData] NVARCHAR(MAX)
);

INSERT INTO @DataSouce ([RecordData])
VALUES ('test some test goes here')
      ,('some numbers go here - 1111122222233333344444444445');


SELECT DS.[RecordID]
      ,RM.[MatchID]
      ,RM.[CaptureValue]
FROM @DataSouce DS
CROSS APPLY [dbo].[fn_Utils_RegexMatches] ([RecordData], '.{1,4}') RM;

Now the data is split. Let's pivot it and show only 6 of the chunks:

SELECT *
FROM
(
    SELECT DS.[RecordID]
          ,RM.[MatchID]
          ,RM.[CaptureValue]
    FROM @DataSouce DS
    CROSS APPLY [dbo].[fn_Utils_RegexMatches] ([RecordData], '.{1,4}') RM
) DS
PIVOT
(
    MAX([CaptureValue]) FOR [MatchID] IN ([0], [1], [2], [3], [4], [5], [6])
) PVT;

Here I use regex function to split the data and PIVOT to create columns and exclude some of the chunks. You can now insert the data in table in order to materialized it and then export it. You can implement such function using the link above or create your own function doing something you need.

score 1 · Accepted Answer · answered Jun 28 '16 at 15:00

You can use a dynamic SQL. Here you are an example you can use to solve your problem:

declare @text nvarchar(max) = N'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.';
declare @len int = 75;

declare @NUMBEROFMEMOS int = CEILING(LEN(@text) / @len);

declare @query nvarchar(max) = N'select ';

declare @loop int = 0;
declare @start int = 1;
declare @memoline int = 1;

while @loop <= @NUMBEROFMEMOS begin
    if @loop > 0 begin
        set @query += N', ';
    end

   set @query += N'substring(''' + @text + N''', ' + cast(@start as nvarchar(max)) + N', ' + cast(@len as nvarchar(max)) + N') as MEMOLINE' + cast(@memoline as  nvarchar(max));

   set @start += @len
   set @loop += 1;
   set @memoline += 1;
end

execute sp_sqlexec @query;

John Cappelletti · Answer 3 · 2016-06-28T15:37:18.947

Here is a cte which will normalize your data. From there you can pivot

Declare @Customer table (FirstName varchar(50),LastName varchar(50),DOB Date,Memo varchar(max))
Insert into @Customer values 
('John','Doe'  ,'1964-07-29','I''ve looked at about 15 different answers on SO but haven''t found this exact situation yet. I''m doing a custom data export and need to export to a data file that will be imported into an older system that needs the data in a specific length/format. I have a "MEMO" column that can have a large amount of text in it. I need to select that value and split it into multiple columns with a FIXED length of 75 chars. For instance, if I have a row with a message that is 185 chars, I need to split that into 3 new columns of 75 chars, MEMO1, MEMO2, MEMO3, with the remaining space in MEMO3 being filled with spaces to equal the 75 chars. The other catch, I can only use up to 18 75-char columns to dump the data into. If it''s longer than 1350 (18x75) chars, the rest gets truncated.'),
('Jane','Smith','1972-03-21','I''m a long-time application dev who is trying to get more involved in the DB side of things. If I were in the C# world, I would just create a method to do a for loop up to NUMBEROFMEMOS and output the data that way. I don''t think that works here though. Thanks in advance!')

Declare @MaxLen int = 75

;with cteBase as (
    Select FirstName,LastName,DOB,Row=1,Memo=substring(Memo,1,@MaxLen) from @Customer
    Union All
    Select h.FirstName,h.LastName,h.DOB,Row=cteBase.Row+1,Memo=substring(h.Memo,((cteBase.Row+0)*@MaxLen)+1,@MaxLen) FROM @Customer h INNER JOIN cteBase ON h.FirstName = cteBase.FirstName and h.LastName = cteBase.LastName where substring(h.Memo,((cteBase.Row+0)*@MaxLen)+1,@MaxLen)<>''
)
--Select * from cteBase Order by LastName,Row
Select FirstName,LastName,DOB
      ,Memo01=max(case when Row=1 then Memo else null end)
      ,Memo02=max(case when Row=2 then Memo else null end)
      ,Memo03=max(case when Row=3 then Memo else null end)
      ,Memo04=max(case when Row=4 then Memo else null end)
      ,Memo05=max(case when Row=5 then Memo else null end)
      ,Memo06=max(case when Row=6 then Memo else null end)
      ,Memo07=max(case when Row=7 then Memo else null end)
      ,Memo08=max(case when Row=8 then Memo else null end)
      ,Memo09=max(case when Row=9 then Memo else null end)
      ,Memo10=max(case when Row=10 then Memo else null end)
      ,Memo11=max(case when Row=11 then Memo else null end)
      ,Memo12=max(case when Row=12 then Memo else null end)
      ,Memo13=max(case when Row=13 then Memo else null end)
      ,Memo14=max(case when Row=14 then Memo else null end)
      ,Memo15=max(case when Row=15 then Memo else null end)
      ,Memo16=max(case when Row=16 then Memo else null end)
      ,Memo17=max(case when Row=17 then Memo else null end)
      ,Memo18=max(case when Row=18 then Memo else null end)
  from cteBase
  Group By FirstName,LastName,DOB
 Order by LastName

The CTE Returns

FirstName   LastName    DOB         Row Memo
John        Doe         1964-07-29  1   I've looked at about 15 different answers on SO but haven't found this exac
John        Doe         1964-07-29  2   t situation yet. I'm doing a custom data export and need to export to a dat
John        Doe         1964-07-29  3   a file that will be imported into an older system that needs the data in a 
John        Doe         1964-07-29  4   specific length/format. I have a "MEMO" column that can have a large amount
John        Doe         1964-07-29  5    of text in it. I need to select that value and split it into multiple colu
John        Doe         1964-07-29  6   mns with a FIXED length of 75 chars. For instance, if I have a row with a m
John        Doe         1964-07-29  7   essage that is 185 chars, I need to split that into 3 new columns of 75 cha
John        Doe         1964-07-29  8   rs, MEMO1, MEMO2, MEMO3, with the remaining space in MEMO3 being filled wit
John        Doe         1964-07-29  9   h spaces to equal the 75 chars. The other catch, I can only use up to 18 75
John        Doe         1964-07-29  10  -char columns to dump the data into. If it's longer than 1350 (18x75) chars
John        Doe         1964-07-29  11  , the rest gets truncated.
Jane        Smith       1972-03-21  1   I'm a long-time application dev who is trying to get more involved in the D
Jane        Smith       1972-03-21  2   B side of things. If I were in the C# world, I would just create a method t
Jane        Smith       1972-03-21  3   o do a for loop up to NUMBEROFMEMOS and output the data that way. I don't t
Jane        Smith       1972-03-21  4   hink that works here though. Thanks in advance!

score 0 · Answer 4 · answered Jun 28 '16 at 15:14

Making sure all your resultant fields are the same length......

DECLARE @NoteText nVARCHAR(1350);
DECLARE @newFieldLength int = 3;  --Yours wull be 75

--GET ORIGINAL TEXT
SET @NoteText = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'; --SET FROM YOURS SOURCE TABLE I GUESS

--MAKE SURE LAST ONE PADS TO THE REQUIRED LENGTH SO ALL FIELDS ARE THE SAME LENGTH
DECLARE @mod int = @newFieldLength - ( LEN(@NoteText) % @newFieldLength );
--SELECT @mod;
WHILE @Mod > 0
BEGIN
  SET @NoteText = @NoteText + ' ';
  SET @mod = @mod - 1;
END

DECLARE @NoOfFields INT;
SELECT @NoOfFields = CEILING(LEN(@NoteText) / @newFieldLength ) + 1;

DECLARE @Loop INT = 0;
DECLARE @dynSQL nVarchar(MAX) = 'SELECT FIRSTNAME, LASTNAME, DOB, ' + CONVERT(nvarchar(4), @NoOfFields) + ' as NUMBEROFMEMOS, '; 
DECLARE @pos INT = 1;
WHILE @Loop < @NoOfFields
BEGIN  
  IF @Loop > 0
  BEGIN
    SET @Pos = (@Loop * @newFieldLength) + 1;
  END;
  SET @dynSQL = @dynSQL + 'SUBSTRING(@NoteText, ' + CONVERT(nvarchar(2), @pos) + ', '  + CONVERT(nvarchar(2), @newFieldLength) +  ') as MEMOLINE_' + CONVERT(nvarchar(2), @loop + 1) + ', ';

  SET @Loop = @Loop + 1;
END

SET @dynSQL = @dynSQL + 'FROM CUSTOMER';
SET @dynSQL = REPLACE( @dynSQL, ', FROM CUSTOMER', ' FROM CUSTOMER ');

--RUN TEH RESULTANT SQL
EXEC @dynSQL

Serpiton · Answer 5 · 2016-06-28T19:39:26.100

If you divide every row of data in a different number of column you will need to create an INSERT statament for every row.
Instead you can always generate all the 18 memo column and do a bulk insert

INSERT INTO [OtherServer.OtherDB.user.table]
SELECT FIRSTNAME, 
       LASTNAME, 
       DOB, 
       LEFT(SUBSTRING(NOTETEXT, 1, 75) + SPACE(75), 75) as MEMOLINE1,
       LEFT(SUBSTRING(NOTETEXT, 76, 150) + SPACE(75), 75) as MEMOLINE2,
       LEFT(SUBSTRING(NOTETEXT, 151, 225) + SPACE(75), 75) as MEMOLINE3,
       ...
       LEFT(SUBSTRING(NOTETEXT, 1276, 1350) + SPACE(75), 75) as MEMOLINE18,
FROM [myServer.myDB.myUser.CUSTOMER]

If there are a lot of lines to export you can work in chunk

Split A Single Field Value Into Multiple Fixed-Length Column Values in T-SQL

5 Answers5

Linked