1

I have to read very large data, approximately 300000 lines, from SQL tables and write it to a text file in my application (.net application and using C#). I have a stored procedure to read that data, and I am using DataAdapter and DataSet in my code to get that data, but I am getting the error

The wait operation timed out

when it executed dataAdapter.Fill(dataSet). I tried to change value of timeout, but it didn't work. I tried different approaches, but nothing seems to work. It works for few lines, but it doesn't work for large data. I can't grab small data because they need whole data in one text file

KenD
  • 5,280
  • 7
  • 48
  • 85
bj1
  • 21
  • 1
  • 7

2 Answers2

1

First and very important change I did-- I changed my DateTime object to shortDate (e.g startDate.ToShortDateString() ), so it deleted the time part from my date, and it helped a lot with my SQL stored procedure with datetime param.

2nd change --- read directly from SqlDataReader and writing directly to text file with Read method, because DataTable will not hold to long file (e.g 420000KB)

using (SqlDataReader rdr = comm.ExecuteReader())
{                               
   while (rdr.Read())
   {
     file.WriteLine(rdr.GetString(0), true);
   }
}

3rd change --I increased my timeout

comm.CommandTimeout = 600; ( e.g for 10min)
marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
bj1
  • 21
  • 1
  • 7
0

See if you can split one query into multiple ones and execute it simultaneously and join the results on completion of all fetches.

Or even using a DataReader will help.

Orion_Eagle
  • 124
  • 5
  • Thx.I already tried DataReader and its giving me same error. Company doesn't wanted to split query. that's the problem – bj1 Jul 10 '15 at 20:53