Exporting data from SQL Server to PostgreSQL

Question

I am using this example to export data from SQL Server to PostgreSQL, when I start the export as 300,000 rows takes 12 minutes, what I can do to speed up this process or you know another way to do it?

string SourceDriver = "Driver={SQL Server Native Client 10.0}";
OdbcConnection SourceConnection = new OdbcConnection(SourceDriver+ ";Server=10.10.10.10;Database=sourceMSSQL;Uid=sa;Pwd=12345;");

string DestDriver = "Driver={PostgreSQL}";
OdbcConnection DestConnection = new OdbcConnection(DestDriver+ ";Server=10.10.10.11;Port=5432;Database=destPostgreSQL;Uid=postgres;Pwd=12345;");

string SourceSql = "SELECT Code, Label, Model, List, Size, Quantity, City, Family,  ExportDate FROM MovPedidosP0";
string DestSql = "INSERT INTO tmp_MovPedidosP0_t (Code, Label, Model, List, Size, Quantity, City, Family,  ExportDate) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?)";

using(OdbcCommand SourceCommand = new OdbcCommand(SourceSql, SourceConnection))
{
    SourceConnection.Open();
    using(OdbcDataReader SourceReader = SourceCommand.ExecuteReader())
    {
        Console.WriteLine("Exporting...");

        DestConnection.Open();

        while(SourceReader.Read())
        {
            using(OdbcCommand DestCommand = new OdbcCommand(DestSql, DestConnection))
            {
                DestCommand.Prepare();
                DestCommand.Parameters.Clear();

                for(int i=0; i<SourceReader.FieldCount; i++)
                {
                    DestCommand.Parameters.AddWithValue("?ID" + (i+1).ToString(), SourceReader[i]);
                }

                DestCommand.ExecuteNonQuery();
                TotalRows++;
            }
        }

        DestConnection.Close();
    }
}

SourceConnection.Close();

You might look into PostgreSql's batch processing: http://stackoverflow.com/questions/758945/whats-the-fastest-way-to-do-a-bulk-insert-into-postgres — NotMe, Mar 21 '13 at 23:07

score 2 · Answer 1 · answered Mar 21 '13 at 23:14

2

Much simpler and probably faster if you export to a text file using SSIS and import with the COPY command.

answered Mar 21 '13 at 23:14

Clodoaldo Neto

118,695
26
233
260

Or even use SSIS to copy directly to the PostgresSQL database. – John Saunders Mar 21 '13 at 23:27
I would use SSIS and COPY but I need to export the information through an application and is an end user who will use it. – Sergio Flores Mar 21 '13 at 23:53

score 0 · Answer 2 · answered Mar 21 '13 at 23:04

0

Try using the native NpgsqlConnection and SqlConnection instead of Odbc connections.

http://npgsql.projects.pgfoundry.org/

http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlconnection.aspx

answered Mar 21 '13 at 23:04

Vedran

10,369
5
50
57

Exporting data from SQL Server to PostgreSQL

2 Answers2