0

I am trying to insert excel data into sql server tables. Each column should be imported with the exact same format that the user wrote in the source Excel.

I am using following query to fetch data

SELECT * FROM OPENROWSET( 'Microsoft.ACE.OLEDB.12.0', 'Excel 12.0 Xml;HDR=YES;IMEX=1;Database=H:\Loadloandata\Test\K3.xlsx', 
'SELECT * FROM [Sheet1$]')

But now in the date column of excel we are receiving some float values( format issues from the users) as shown below

enter image description here

Because of the invalid data, the OLE provider convert the all other dates to float values in corresponding SQL table colums (float values corresponding to each date).If a date column is automatically cast to float I won't be able to know the original format of the data in the excel file, so all columns should be imported as varchar.

enter image description here

How can i prevent this datatype conversion? Based on google search i have used IMEX=1 in connection string to retrieve data for mixed data columns. But it is not working !!

AcAnanth
  • 765
  • 3
  • 19
  • 53
  • 1
    Reason number 10 billion why Excel is bad as a data source... I would recommend changing the cell types for that column in Excel to proper dates or text. – Jacob H Sep 25 '19 at 13:49
  • @JacobH we already informed the users but sometimes we are receiving the same..Any way need to handle since data coming as bulk..Or any other alternative way? – AcAnanth Sep 25 '19 at 14:04
  • The integer part of the float is the number of days since 30/12/1899 (excel) - SQL Server uses 01/01/1900 as its number of days, you could therefore convert by using Select CAST(exceldatecolumn - 2 as SmallDateTime). See https://stackoverflow.com/questions/13850605/convert-excel-date-serial-number-to-regular-date – Steve Ford Oct 07 '19 at 12:38
  • So did it work for you or not? I would like to see some update from your side. – tukan Oct 10 '19 at 07:37
  • @tukan it is not working...It is still converting date values as float – AcAnanth Oct 14 '19 at 13:10
  • Could you provide source data and the converted excel? – tukan Oct 14 '19 at 18:36
  • What settings did you use? I would need some details, otherwise it is hard to help you. – tukan Oct 15 '19 at 06:47

2 Answers2

0

I think that you should get the data types from the SQL server table first to create the recordset, rather than letting Excel decide the datatypes from the Sheet. I believe Excel decides the data types by the first row, so in your case, it assumed Funded data was an integer, then casts any following strings into that data type.

Here is a full function that I use

This code is modified by me from the original source, which is mentioned in the comment. I made changes to deal with errors better.

    Function ExportRangeToSQL(ByVal sourcerange As Range, _
    ByVal conString As String, ByVal table As String, _
    Optional ByVal beforeSQL = "", Optional ByVal afterSQL As String) As String

    'https://www.excel-sql-server.com/excel-sql-server-import-export-using-vba.htm

    ' Object type and CreateObject function are used instead of ADODB.Connection,
    ' ADODB.Command for late binding without reference to
    ' Microsoft ActiveX Data Objects 2.x Library
    ' ADO API Reference
    ' https://msdn.microsoft.com/en-us/library/ms678086(v=VS.85).aspx
    ' Dim con As ADODB.Connection

    On Error GoTo Finalise ' throw friendly user connection error

    Dim con As Object
    Set con = CreateObject("ADODB.Connection")

    con.ConnectionString = conString
    con.Open


    Dim cmd As Object
    Set cmd = CreateObject("ADODB.Command")

    ' BeginTrans, CommitTrans, and RollbackTrans Methods (ADO)
    ' http://msdn.microsoft.com/en-us/library/ms680895(v=vs.85).aspx

    Dim level As Long
    level = con.BeginTrans

    cmd.CommandType = 1             ' adCmdText
    If beforeSQL > "" Then
        cmd.CommandText = beforeSQL
        cmd.ActiveConnection = con
        cmd.Execute
    End If

    ' Dim rst As ADODB.Recordset
    Dim rst As Object
    Set rst = CreateObject("ADODB.Recordset")

    With rst
        Set .ActiveConnection = con
        .Source = "SELECT * FROM " & table
        .CursorLocation = 3         ' adUseClient
        .LockType = 4               ' adLockBatchOptimistic
        .CursorType = 0             ' adOpenForwardOnly
        .Open

        ' Column mappings

        Dim tableFields(100) As Integer
        Dim rangeFields(100) As Integer

        Dim exportFieldsCount As Integer
        exportFieldsCount = 0

        Dim col As Integer
        Dim index As Variant

        For col = 0 To .Fields.Count - 1
            index = 0
            index = Application.Match(.Fields(col).Name, sourcerange.Rows(1), 0)
            If Not IsError(index) Then
                If index > 0 Then
                    exportFieldsCount = exportFieldsCount + 1
                    tableFields(exportFieldsCount) = col
                    rangeFields(exportFieldsCount) = index
                End If
            End If
        Next

        If exportFieldsCount = 0 Then
            Err.Raise 513, , "Column mapping mismatch between source and destination tables"
        End If

        ' Fast read of Excel range values to an array
        ' for further fast work with the array

        Dim arr As Variant
        arr = sourcerange.Value

        ' The range data transfer to the Recordset

        Dim row As Long
        Dim rowCount As Long
        rowCount = UBound(arr, 1)

        Dim val As Variant

        For row = 2 To rowCount
            .AddNew
            For col = 1 To exportFieldsCount
                val = arr(row, rangeFields(col))
                If IsEmpty(val) Then
                Else
                    .Fields(tableFields(col)) = val
                End If
            Next
        Next

        .UpdateBatch
    End With

    rst.Close
    Set rst = Nothing

    If afterSQL > "" Then
        cmd.CommandText = afterSQL
        cmd.ActiveConnection = con
        cmd.Execute
    End If



Finalise:
If con.State <> 0 Then
    con.CommitTrans
    con.Close
End If

Set cmd = Nothing
Set con = Nothing

' Raise appropriate custom errors

Select Case Err.Number
    Case -2147217843
        Err.Raise 513, , "Issue connecting to SQL server database - please check login credentials"

    Case -2147467259
        If InStr(1, Err.Description, "Server does not exist") <> 0 Then
            Err.Raise 513, , "Could not connect to SQL server, please check you are connected to the local network (in the office or on VPN)"
        Else
             Err.Raise 513, , "Issue connecting to SQL server database" & vbNewLine & Err.Description
        End If
    Case -2147217900
        If InStr(1, Err.Description, "'PK_XL_Eng_Projects_QuoteRef'") <> 0 Then
             Err.Raise 513, , "Quote already uploaded for this QuoteRef and Upload Time, please wait a minute before trying again" & vbNewLine & vbNewLine & Err.Description
         Else
            Err.Raise Err.Number, , Err.Description
         End If
    Case 0
        ' do nothing no error
    Case Else
        ' re raise standard error
         Err.Raise Err.Number, , Err.Description

End Select


End Function
James Anderson
  • 181
  • 2
  • 8
-1

Is there a reason why you using SSIS? I think that is best suited for the job.

Anyways, back to your issue. IMEX=1 is not enough. What you need is to check the registry entries

You need set TypeGuessRows and ImportMixedTypes within this registry path (this is for 32-bit office!):

HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Office\12.0\Access Connectivity Engine\Engines\Excel

TypeGuessRows = 0 (the default is 8)

ImportMixedTypes = Text

What does TypeGuessRows do?

It tries to guess data type based on the number of rows defined. The default values is 8. That means it will check 8 rows to see what data type should be used. If you want to the engine to scan all the rows put 0 there. There is a catch, however, if your spreadsheet large, you could pay heavy performance penalty for such setting.

What does ImportMixedTypes do?

This is where your IMEX setting comes into the game. There are 3 possible values for IMEX setting 0, 1, 2:

  • 0 is Export mode
  • 1 is Import mode
  • 2 is Linked mode (full update capabilities)

Only when setting IMEX=1 the registry value is honored. - with the default setting being ImportMixedTypes=Text. In any other value (0, 2) the value in registry is checked, if it is valid, but it does not influence the outcome. (you will get an error if invalid)

There are two valid values for ImportMixedTypes:

ImportMixedTypes=Text
ImportMixedTypes=Majority Type

The Majority Type is rarely used. What it does it counts type of each column and the majority type is then used for the whole column. The Text type will limit the row size to 255 characters, if you want to use more characters then Majority Type must be used and the majority must use more than 256 characters.

tukan
  • 17,050
  • 1
  • 20
  • 48