Understanding TPL Dataflow, Blocks, and Continuation Tasks

Question

I am working on a project where a need arose that is a perfect scenario for TPL Dataflow. Having relatively limited experience with it (and what I do have was done some time ago), I have been brushing up on it by reading Microsoft's documentation as well as articles I can find online.

Having done that, I built my code to chain together a series of blocks (mostly TransformBlock and ending with an ActionBlock doing something like this:

var block1 = new TransformBlock<T, U>(async input => {});
var block2 = new TransformBlock<U, V>(async input => {});
var block3 = new ActionBlock<V>(async input => {});

block1.LinkTo(block2);
block2.LinkTo(block3);

foreach(var item in items)
{
   await block1.SendAsync(item);
}
block1.Complete();
await block3.Completion;

Someone in an article (which I can't find) suggested that there should be continuation tasks in the pipeline to mark blocks as complete. This is the code they provided for that.

// Create the continuation tasks in the pipeline that marks each block as complete.
await block1.Completion.ContinueWith(t =>
{
    if (t.IsFaulted) { ((IDataflowBlock)block2).Fault(t.Exception); }
    else { block2.Complete(); }
});
await block2.Completion.ContinueWith(t =>
{
    if (t.IsFaulted) { ((IDataflowBlock)block3).Fault(t.Exception); }
    else { block3.Complete(); }
});

I will admit that I don't fully understand what this code is doing and whether it is even needed. When I attempt to run this in the code I just wrote, the code hangs on the first ContinueWith and it never makes it to running the pipeline.

I would appreciate additional explanation since I want to get a better understanding of the nuances of what is going on here.

You should not be asking multiple separate questions per question. Additionally, explaining the entirety of some concept is Too Broad. You should be asking a *single, specific* question. — Servy, Dec 13 '17 at 19:53
@Servy I removed my first question. How would you suggest asking the second part? I provided a piece of code that I do not understand why someone said it was important (and the article didn't explain it either). At this point, I haven't found/read anything that helped me to understand it better. — JasCav, Dec 13 '17 at 20:03
You didn't find *anything* about how completion of blocks works, and you need people to explain you *all* of the nuances of how competition works? For the first part, there's plenty of info about how completion of blocks works, for the second, again, that's Too Broad. — Servy, Dec 13 '17 at 20:33
I didn't say I didn't find anything at all...I said I didn't understand that piece of code from the article. I provided specific code. @JSteward provided a good answer, I believe and his link provides more detail to the specific code above so that helps. Perhaps my Google-fu is completely failing. I will work to craft a clearer question in the future, but someone was able to point me in the right direction. Thank you. — JasCav, Dec 13 '17 at 20:35

JSteward · Accepted Answer · 2017-12-13T20:22:41.727

All you need to do for a linear pipeline is to PropagateCompletion. That option propagates Completion as well as Faults whose exception is then attached to the final Completion Task:

var linkOptions = new DataflowLinkOptions() { PropagateCompletion = true };
block1.LinkTo(block2, linkOptions);
block2.LinkTo(block3, linkOptions);

The continuations are unnecessary. But if you have pipeline that distributes to multiple blocks you'll need to handle completion and fault propagation yourself as shown here.

the code hangs on the first ContinueWith

That happens because you await the continuation, so if you do that before calling Complete() then block1 will never complete.

Understanding TPL Dataflow, Blocks, and Continuation Tasks

1 Answers1