Using Ienumerable.TakeWhile but only returns one set of results

Question

First off I want to apologize if my code is bad or if my description is poor. This is one of my first times working with C# threading/tasks. What I'm trying to do in my code is to go through a list of names and for each 50 names in the list, start a new task and pass off those 50 names to another method that will perform calculation heavy methods on the data. My code only works for the first 50 names in the list and it returns 0 results for every other time and I can't seem to figure out why.

public static async void startInitialDownload(string value)
    {
        IEnumerable<string> names = await Helper.getNames(value, 0);
        decimal multiple = names.Count() / 50;
        string[] results;
        int num1 = 0;
        int num2 = 0;

        for (int i = 0; i < multiple + 1; i++)
        {
            num1 = i * 50;
            num2 = (50 * (i + 1));
            results = names.TakeWhile((name, index) => index >= num1 && index < num2).ToArray();
            Task current = Task.Factory.StartNew(() => getCurrentData(results));
            await current.ConfigureAwait(false);
        }
    }

Guffa · Accepted Answer · 2014-05-18T21:20:29.800

Realise the enumerable into a list, so that it will be calculated once, not each iteration in the loop. You can use the Skip and Take methods to get a range of the list:

public static async void startInitialDownload(string value) {
  IEnumerable<string> names = await Helper.getNames(value, 0);
  List<string> nameList = names.ToList();
  for (int i = 0; i < nameList.Count; i += 50) {
    string[] results = nameList.Skip(i).Take(50).ToArray();
    Task current = Task.Factory.StartNew(() => getCurrentData(results));
    await current.ConfigureAwait(false);
  }
}

Or you can add items to a list, and execute it when it has the right size:

public static async void startInitialDownload(string value) {
  IEnumerable<string> names = await Helper.getNames(value, 0);
  List<string> buffer = new List<string>();
  foreach (string s in names) {
    buffer.Add(s);
    if (buffer.Count == 50) {
      Task current = Task.Factory.StartNew(() => getCurrentData(buffer.ToArray()));
      await current.ConfigureAwait(false);
      buffer = new List<string>();
    }
  }
  if (buffer.Count > 0) {
    Task current = Task.Factory.StartNew(() => getCurrentData(buffer.ToArray()));
    await current.ConfigureAwait(false);
  }
}

score 0 · Answer 2 · answered May 18 '14 at 21:04

The name TakeWhile suggests that it only takes entries while the condition is true. So if it starts off by reading an entry for which the condition is false, it never takes anything.

So the first loop, you're starting with num1 = 0. So it reads entries from num1 to num2.

The second loop, you're starting with num1 being 51. So it starts reading again ... and the first entry it hits, the condition is false, so it stops.

You might try using Where, or by using Skip before hand.

The tl;dr; of it: I don't think your problem has anything to do with parallel tasks, I think it's due to using the wrong LINQ method to pull the names you want to use.

score 0 · Answer 3 · edited May 23 '17 at 10:25

As I understand it from Stephen Cleary's response to a similar (though not identical) question, you don't need to use ConfigureAwait() there.

Here's the link in question: on stack overflow

And here's what I would do instead with the last two lines of your for loop:

Task.Factory.StartNew(() => getCurrentData(results));

That's it. By using the factory, and by not awaiting, you are letting that task run on its own (possibly on a new thread). Provided that your storage is all thread safe (see: System.Collections.Concurrent btw) then you should be all set.

Caveat: if you aren't showing us what lies after the await then your results may vary.

score 0 · Answer 4 · answered May 18 '14 at 21:07

its not a direct solution, but it might work.

public static IEnumerable<T[]> MakeBuckets<T>(IEnumerable<T> source, int maxSize)

    {
        List<T> currentBucket = new List<T>(maxSize);

        foreach (var s in source)
        {
            currentBucket.Add(s);
            if (currentBucket.Count >= maxSize)
            {
                yield return currentBucket.ToArray();
                currentBucket = new List<T>(maxSize);

            }

        }
        if(currentBucket.Any())
            yield return currentBucket.ToArray();
    }

later you can iterate through the result of the MakeBucket function.

Using Ienumerable.TakeWhile but only returns one set of results

4 Answers4