When I am importing data as text from excel, it is importing as scientific notation to SQL, even when I can see at all steps during the process the data appears as text without scientific notation. Here is a snippet of the original file:
Utilizing the ExcelDataReader
3rd party code (https://github.com/ExcelDataReader/ExcelDataReader), I am able to extract a DataTable from the excel file and read it. When reading the data, the column and values show up as I suspect from a scratch console app.
When I run the code responsible for sending to SQL, I can see in the SQL profiler the SQL it is executing. I can copy it and execute it separately, which then imports the data as I would expect. Here is a sample.
declare @p1 dbo.ImportDataTableTest
insert into @p1 values(1623157535)
insert into @p1 values(1690205652)
insert into @p1 values(1690205654)
insert into @p1 values(1623110539)
insert into @p1 values(1623165689)
insert into @p1 values(1623165689)
insert into @p1 values(1623171209)
insert into @p1 values(1623128536)
insert into @p1 values(1623172096)
exec InsertStagingData @ImportDataTable=@p1
My problem shows up here after running the application as normal. This is the result in the table:
InvNo StagingID
3.22029e+009 6491
3.22028e+009 6492
3.22028e+009 6493
3.22028e+009 6494
3.22029e+009 6495
Here is the scratch app and sql code I am using for this example:
C#
namespace ConsoleApp1
{
class Program
{
static void Main(string[] args)
{
string fileName = @"path to the file";
ExcelDataWrapper wrapper = new ExcelDataWrapper();
ExcelFile file = wrapper.GetExcelData(fileName);
DataTable dt = file.ExcelData.Tables[0];
using (var conn = new SqlConnection("our connection string"))
using (var command = new SqlCommand("InsertStagingData", conn)
{
CommandType = CommandType.StoredProcedure
})
{
conn.Open();
//This parameter name would be passed in from our steps table and would allow
//for different values to be imported per database
command.Parameters.Add(new SqlParameter("@ImportDataTable", dt));
command.ExecuteNonQuery();
}
}
}
}
namespace ExcelDataReaderWrapper
{
public class ExcelDataWrapper
{
public ExcelFile GetExcelData(string pathToFile)
{
DataSet ds;
ds = Read(pathToFile);
string fileName = Path.GetFileName(pathToFile);
ExcelFile dataTableWithFileName = new ExcelFile { FileName = fileName, ExcelData = ds };
return dataTableWithFileName;
}
private DataSet Read(string file)
{
System.Text.Encoding.RegisterProvider(System.Text.CodePagesEncodingProvider.Instance);
using (var stream = File.Open(file, FileMode.Open, FileAccess.Read))
{
if (file.EndsWith(".xls"))
{
using (IExcelDataReader reader = ExcelReaderFactory.CreateBinaryReader(stream))
{
var conf = new ExcelDataSetConfiguration
{
ConfigureDataTable = _ => new ExcelDataTableConfiguration
{
UseHeaderRow = true
}
};
return reader.AsDataSet(conf);
}
}
if (file.EndsWith(".xlsx"))
{
using (IExcelDataReader reader = ExcelReaderFactory.CreateOpenXmlReader(stream))
{
var conf = new ExcelDataSetConfiguration
{
ConfigureDataTable = _ => new ExcelDataTableConfiguration
{
UseHeaderRow = true
}
};
return reader.AsDataSet(conf);
}
}
return null;
}
}
}
}
SQL:
/****** Object: UserDefinedTableType [dbo].[ImportDataTableTest] Script Date: 11/19/2019 12:18:18 PM ******/
CREATE TYPE [dbo].[ImportDataTableTest] AS TABLE(
[InvNo] [nvarchar](255) NULL
)
GO
CREATE Procedure [dbo].[InsertStagingData]
@ImportDataTable ImportDataTableTest readonly
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
Insert into StagingPOS([InvNo])
Select InvNo
from @ImportDataTable
END
GO
Is there some solution I am missing that gets the data as I see it from the application and the profiler to the table correctly as a nvarchar
field? I tried casting the values, but the import looses some of the precision that I need.
EDIT 1:
I followed the link provided by @pcalkins. This would appear to solve the problem as you can see here:
ALTER Procedure [dbo].[InsertStagingData]
@ImportDataTable ImportDataTableTest readonly
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
Insert into StagingPOS([InvNo])
select Case When InvNo like '%e+%' or InvNo like '%e-%' then replace(cast(cast(cast([InvNo] as real) as money) as nvarchar(255)),'.00','')
Else InvNo
End
from @ImportDataTable
END
InvNo StagingID
3220290048 8435
3220280064 8436
3220280064 8437
3220280064 8438
But when I query the values in my excel file to the results in the table, nothing comes up. The values appear rounded at different points in the number.