2

If I got it correctly, JoinBlock<T1, T2> is the "WaitAll" of TPL Dataflow: when you have one T1 and one T2, it builds a Tuple<T1, T2> and passes it to the next block.

Is there a "ChoiceBlock<T1, T2>", the "WaitAny" of Dataflow, where either a Block<T1> or a Block<T2> are executed next?

Something like:

var choice = new ChoiceBlock<string, int>
res1.LinkTo(choice.Target1);
res2.LinkTo(choice.Target2);

choice.LinkTo(intConsumer);
choice.LinkTo(stringConsumer);

EDIT: this answer is almost what I want, I'd just like to know if there is a native/more elegant solution when you have consumers of different types, and avoid all the casting and typeof(T) checks.

EDIT2: Just to make it even clearer: this works

            var stringCast = new TransformBlock<object,string>(o => (string)o);
            var stringTarget = new ActionBlock<string>(m_printAction);
            stringCast.LinkTo(stringTarget);

            var eventCast = new TransformBlock<object, Event>(o => (Event)o);
            var eventTarget = new ActionBlock<Event>(m_addEventAction);
            eventCast.LinkTo(eventTarget);

            var forwarder = new BufferBlock<object>();
            forwarder.LinkTo(stringCast, item => item is string);
            forwarder.LinkTo(eventCast, item => item is Event);

            forwarder.LinkTo(DataflowBlock.NullTarget<object>());

            m_messages.LinkTo(forwarder);
            m_events.LinkTo(forwarder);

But it's ugly and inefficient. Is there something better suited?

The old CCR (concurrency and coordination runtime) used to have both Join and Choice, but I can't find Choice in TPL Dataflow. Am I missing something?

Community
  • 1
  • 1
Lorenzo Dematté
  • 7,638
  • 3
  • 37
  • 77
  • A dataflow is a pipeline of steps that process a stream of records. What you ask is how to *join* multiple message streams. There are multiple such blocks, eg the JoinBlock – Panagiotis Kanavos Mar 03 '17 at 09:48
  • Nope, I am asking how to select one of the streams. Like the "plus"/"choice" operator on CPS or Pi calculus, but based on types. – Lorenzo Dematté Mar 03 '17 at 09:49
  • You are using the wrong terms and mental model then. There is no wait all or wait any. It's streams of messages and their combinations. The equivalent is the SSIS dataflow, or the Powershell pipeline. In any case, `choice` is unrelated to the TPL's `WaitAny()`. The comparison simply confuses things – Panagiotis Kanavos Mar 03 '17 at 09:53
  • 1
    You can specify a predicate in `LinkTo` to transfer only messages that fulfill specific criteria to a block. You don't need a "choice" block. Take care to cover *all* options though, or provide an unfiltered link, otherwise messages will get stuch in the output buffer of the preceding block – Panagiotis Kanavos Mar 03 '17 at 09:53
  • I know a thing or two about the programming model :) having written a couple of libraries myself. As I wrote, I am looking for the "plus" operator of pi calculus and the like. I was just drawing a parallel here to try and make it clearer what I'm asking for: Join : WaitAll = Choice : WaitAny, but I can't find a good Choice implementation in TDF. It's strange because its ancestor, the CCR, had it. – Lorenzo Dematté Mar 03 '17 at 10:10
  • Then explain what you want in dataflow terms, not the Pi calculus. This *causes* confusion instead of explaining anything. You don't have to know about it to use the TPL dataflow or any other kind of pipeline processing. As for `WaitAll`, `WaitAny` they have a very specific meaning in the TPL - there are `Task.WhenAll`, `Task.WhenAny` which don't block and `Task.WaitAll` which blocks. – Panagiotis Kanavos Mar 03 '17 at 10:16
  • As for your edit - it *is* ugly, mainly because `object` is used. Why do that? You can write your own generic extension method to connect multiple blocks with predicates without resorting to `object`.. Where do the `object` values come from? – Panagiotis Kanavos Mar 03 '17 at 10:21

1 Answers1

4

Aside from some really creative use of Dataflowblock.Choose MSDN.

There really isn't any choice being made in your example:

var choice = new ChoiceBlock<string, int>
res1.LinkTo(choice.Target1);
res2.LinkTo(choice.Target2);

choice.LinkTo(intConsumer);
choice.LinkTo(stringConsumer);

The preceding block would have to not know what type it was going to output, leading to the ugly object type paramter or you would link two blocks unnecessarily into one. Assuming the latter what's wrong with two distinct typed pipelines?

var buffer1 = new BufferBlock<int>();
var buffer2 = new BufferBlock<string>();
var transform1 = new TransformBlock<int, int>((x) => x);
var transform2 = new TransformBlock<string, string>((x) => x);
buffer1.LinkTo(transform1);
buffer2.LinkTo(transform2);

Is there a "ChoiceBlock", the "WaitAny" of Dataflow, where either a Block<T1> or a Block<T2> are executed next?

At anytime, if output becomes available at either source, i.e. buffer, its made available to either Block<int> or Block<string>. Does this not fullfill the WaitAny implied contract in a typed way?

Not necessarily an answer to the OP question but too long for a comment.

JSteward
  • 6,833
  • 2
  • 21
  • 30
  • `Dataflowblock.Choose` was exactly what I was looking for! But you are right, the ugly object parameter is there because I have linked two blocks in one (2 to 1 ---other steps--- 1 to 2 (choose)) and didn't know what was coming in. A better solution is to refactor and leave the two pipelines separated. – Lorenzo Dematté Mar 03 '17 at 15:20