how to read big text files in C#?

Question

I have a win Form and I want to read a text file and show it in list Box but there is problem when i try to load file with 1-3mb size after a few seconds its throw this exception:

Managed Debugging Assistant 'ContextSwitchDeadlock' : 'The CLR has been unable to transition from COM context 0x149cf10 to COM context 0x149ce58 for 60 seconds. The thread that owns the destination context/apartment is most likely either doing a non pumping wait or processing a very long running operation without pumping Windows messages. This situation generally has a negative performance impact and may even lead to the application becoming non responsive or memory usage accumulating continually over time. To avoid this problem, all single threaded apartment (STA) threads should use pumping wait primitives (such as CoWaitForMultipleHandles) and routinely pump messages during long running operations.'

and here is my code:

private void Button7_Click(object sender, EventArgs e)
    {
        openFileDialog = new OpenFileDialog();
        if (openFileDialog.ShowDialog() == DialogResult.OK)
        {
            StreamReader sr = new StreamReader(openFileDialog.FileName);
            var emails = sr.ReadToEnd();
            foreach (var items in emails)
            {
                aFlistBoxEmail.Items.Add(items);
            }
        }
    }

also i use this solution but didnt help: enter link description here

`sr.ReadToEnd()` returns a `string`. Your `foreach` loops over that string (as a charachterarray), adding all every single _character_ to the Items collection. That means you are doing that a million times for a 1MB file. Is that really your intention? — oerkelens, Jul 07 '19 at 09:21
My solution? I have no clue what you have in your files, or what you want to do with that content. Maybe you want to `ReadAllLines`, maybe you want to split the file content in some other way? — oerkelens, Jul 07 '19 at 10:33

score 0 · Answer 1 · answered Jul 07 '19 at 09:01

Your click handler executes on the UI thread, which also happens to run the message pump for your application. Since your handler takes so long to run, it makes your application nonresponsive to UI and system events for a period that is probably unacceptable.

You can remedy the problem by making your logic async.

private async Task Button7_Click(object sender, EventArgs e)
{
    openFileDialog = new OpenFileDialog();
    if (openFileDialog.ShowDialog() == DialogResult.OK)
    {
        StreamReader sr = new StreamReader(openFileDialog.FileName);
        var emails = await sr.ReadToEndAsync();
        foreach (var items in emails)
        {
            aFlistBoxEmail.Items.Add(items);
        }
    }
}

score 0 · Answer 2 · answered Jul 07 '19 at 09:07

When there are long running process, then automatically the application will go into non responsive mode , because the long running process is inside the UI thread, Its better to use the asyn programming using await & async , this will help to run the long running activities into a seperate thread and it wont break the UI thread. My sample code is given below.

private async void Button7_Click(object sender, EventArgs e)
{
    openFileDialog = new OpenFileDialog();
    if (openFileDialog.ShowDialog() == DialogResult.OK)
    {
        StreamReader sr = new StreamReader(openFileDialog.FileName);
        var emails =  await sr.ReadToEndAsync();
        foreach (var items in emails)
        {
            aFlistBoxEmail.Items.Add(items);
        }
    }
}

The the file reading part can be moved to another function so that it can give more modulatiry.

ty for your sample code. i used yours but still nothing its stucks and app doesnt respond... is there any other ways to read big files? — , Jul 07 '19 at 09:19

score 0 · Answer 3 · answered Jul 07 '19 at 09:34

Why are you doing this exactly? your code adds every single character in the text file as a separate entry in the list box, with a 3 MB file it will be a lot of entries, don't you need to parse the text to a specific value or a class first? if you're trying to read emails from a file split the file like this for example string.Split('\n'); in a way you separate each email and then parse them using the MailAddress Class from System.Net.Mail.MailAddress

Slai · Accepted Answer · 2019-07-07T11:00:02.513

0

You can try reading all lines at once :

private async void Button7_Click(object sender, EventArgs e)
{
    openFileDialog = new OpenFileDialog();
    if (openFileDialog.ShowDialog() == DialogResult.OK)
        aFlistBoxEmail.DataSource = File.ReadAllLines(openFileDialog.FileName);
}

edited Jul 07 '19 at 11:00

answered Jul 07 '19 at 09:34

Slai

22,144
5
45
53

Incoming nitpick: No, the problem is not related to the StreamReader not being closed (the StreamReader not being closed would not stall the message-pumping UI thread...). The first comment underneath the question pinpoints the actual cause of the problem. Still, your answer contains the/a solution to the problem ;-) – Jul 07 '19 at 10:38
thanks @awrashpk. I skimmed the question without thinking much. – Slai Jul 07 '19 at 11:06

score 0 · Answer 5 · answered Jul 07 '19 at 13:48

it doesn't necessarily mean you have an issue but it can cause issues since you're working on the main thread of the UI - so your app will be stuck until the reader finishes it's work and since it's a heavy file you're getting this error. you can switch this off in:

Debug > windows > Exceptions settings > Managed Debugging Assistants (expand) > disable "ContextSwitchDeadlock"

it's a good idea to put this task in a thread or a task and make sure you have try catch INSIDE the method as well in case it fails (or a using statement) and meanwhile (until the file reader finishes) you can display on the UI a message saying "app is busy" or something similar.

Thread thread = new Thread(Butoon_Click(sender, ev));
thread.IsBackground = true;

how to read big text files in C#?

5 Answers5