How to save Large Data to file

Question

I have list of maybe 50,000 entries that are populated in datagrid in wpf. Now I want to save the data in the list to a file that may be text, or preferably CSV. As list is too big. There is a problem that my implemented method that may be simple text file writing or the method to copy the contents from the datagrid to clipboard and then back to string, and then that string to file using StreamReader. It consumes approx 4-5 minutes even it is in background worker.

Is there any way that I can save huge list to file quickly?

I am using DataGrid in WPF

CODE

 dataGrid1.SelectAllCells();
            dataGrid1.ClipboardCopyMode = DataGridClipboardCopyMode.IncludeHeader;
            ApplicationCommands.Copy.Execute(null, dataGrid1);
   String result = (string)Clipboard.GetData(DataFormats.CommaSeparatedValue);


///Never reach to step Below thread stays on above line
                dataGrid1.UnselectAllCells();
                Clipboard.Clear();
                StreamWriter file = new System.IO.StreamWriter(SavePageRankToPDF.FileName);
                file.WriteLine(result);
                file.Close();

Can you show us the code you currently use that's taking so long? If you step through the code in a debugger, what's the most time-consuming step? — David, Mar 22 '11 at 20:03
Are you using the datagrid as a standalone object, or is it bound to a database? — Albert Perrien, Mar 22 '11 at 20:04
standalone data comes from custom List.I was interested in using list rather than using this approach — Afnan Bashir, Mar 22 '11 at 20:09
So the data is in your list(dataset?) and you point to it with your grid? — Albert Perrien, Mar 22 '11 at 20:29
You can check this https://gist.github.com/riyadparvez/4467668 — user, Mar 12 '13 at 07:19

score 2 · Accepted Answer · edited May 23 '17 at 11:53

2

Instead of using the clipboard, why not iterate through the datatable and build the csv file.

Update

Here are some examples:

Convert DataTable to CSV stream

Converting DataSet\DataTable to CSV

edited May 23 '17 at 11:53

Community

1
1

answered Mar 22 '11 at 20:49

any example for creating csv with ietration – Afnan Bashir Mar 22 '11 at 21:00

score 0 · Answer 2 · answered Mar 22 '11 at 20:08

0

One thing that will help is to not load ALL of your data into the datagrid when using it for display purposes. It'd be a good idea to use paging: only load the data into the datagrid that will be needed for calculations or display purposes. If the user wants to see/use more data, go back to your data source and get more of the data. Not only will your app run faster, you'll use much less memory.

answered Mar 22 '11 at 20:08

MonkeyWrench

1,809
2
24
48

yes i have managed the data in the and datagrid takes approx 3-4 sec to display all data but it is file saving which i suffer the most – Afnan Bashir Mar 22 '11 at 20:12
problem is not with the grid.Problem is with the file writing.Is there any way in which i can eliminate use of clipboard memory and use List to write files – Afnan Bashir Mar 22 '11 at 20:22

How to save Large Data to file

2 Answers2