4

The title of my question says it all.

I am looking for a TPL dataflow block that doesn't need an input.

Right now I am using a transform block but it's input is unused.

Bilal Fazlani
  • 6,727
  • 9
  • 44
  • 90
  • No. Why do you need it too? Maybe we can create something like that. – i3arnon Jul 05 '16 at 08:44
  • My first block pulls data from a queue. It doesn't need any input to pull data from the queue. – Bilal Fazlani Jul 05 '16 at 08:45
  • 1
    @BilalFazlani actually, it does have an input. The queue itself. A TransformManyBlock can receive the queue as input and return its contents as output. Or you could replace your queue with a BufferBlock (which is a queue as well). Or you could just use a loop that pops messages from the queue and posts them to the first block – Panagiotis Kanavos Jul 05 '16 at 08:46
  • @PanagiotisKanavos not really. The queue is fixed for all messages. There is no need for me to parameterize the queue because it never changes. – Bilal Fazlani Jul 05 '16 at 08:48
  • @BilalFazlani global statics are a bad idea, and that's what this queue is if you hard-code it inside the block. Besides, why create a custom block when there is one that can do the job for you already? Finally, how can you test your mesh if the queue is hard-coded inside the first block? – Panagiotis Kanavos Jul 05 '16 at 08:49
  • Yes, I am using an Transform manyblock right now. But its like `TransformManyBlock>` and i just pass 0 – Bilal Fazlani Jul 05 '16 at 08:50
  • The queue is being injected from IoC. – Bilal Fazlani Jul 05 '16 at 08:51
  • May be you are right – Bilal Fazlani Jul 05 '16 at 08:51
  • 1
    Pass a reference to the queue then! In the delegate itself, use `yield return` to push each message immediately, eg `queue=>{ while (!queue.Length==0){var msg=queue.Pop(); yield return msg;}}` – Panagiotis Kanavos Jul 05 '16 at 08:51
  • I will just parameterize the queue if there is no other block that can give an output without input – Bilal Fazlani Jul 05 '16 at 08:51
  • Its an aws SQS queue btw :) but I can parameterize it. – Bilal Fazlani Jul 05 '16 at 08:53
  • 1
    @BilalFazlani you *HAVE* to parameterize it in this case to be able to test against a staging environment. I have quite a few stories about tests that accidentally used a production queue URL. In this case, the input can be the SQS URL (or the client class if you use an IoC container) – Panagiotis Kanavos Jul 05 '16 at 08:55
  • I will parameterize because it seems like a good idea. it but I don't HAVE to. that's because I am registering a real queue in the application start. In the test project I am overriding that registration with a fake queue. And our staging/testing environment don't have access to prod queue. But parameterizing make sense here given that there is no block that suits my requirement. using IoC is in a way parameterizing through constructor. – Bilal Fazlani Jul 05 '16 at 09:01

2 Answers2

5

I would build a block like this from a BufferBlock<T>: the method accepts a delegate that presents the ITargetBlock<T> side of the block and returns the ISourceBlock<T> side of it. This way, the delegate can send input to the block, but from the outside, it looks like a block that only produces output.

The code:

public static ISourceBlock<T> CreateProducerBlock<T>(
    Func<ITargetBlock<T>, Task> producer,
    int boundedCapacity = DataflowBlockOptions.Unbounded)
{
    var block = new BufferBlock<T>(
        new ExecutionDataflowBlockOptions { BoundedCapacity = boundedCapacity });

    Task.Run(async () =>
    {
        try
        {
            await producer(block);

            block.Complete();
        }
        catch (Exception ex)
        {
            ((IDataflowBlock)block).Fault(ex);
        }
    });

    return block;
}

Example usage:

var producer = CreateProducerBlock<int>(async target =>
{
    await target.SendAsync(10);
    await target.SendAsync(20);
});

ITargetBlock<int> consumer = …;

producer.LinkTo(consumer);
svick
  • 236,525
  • 50
  • 385
  • 514
  • I was looking for an example of ISourceBlock implementation, but this is simple and elegant! – Guge Sep 17 '16 at 09:58
0

Sometimes it is easiest to just use a throwaway bool as the input for a TransformManyBlock, and .Post(true) to kick off your pipeline.

amonroejj
  • 573
  • 4
  • 16