2

I'm using a System.IO.Pipelines.Pipe in .NET 5 to transfer data from a stream to a reader, and now I'm thinking I'd like to have several independent readers that would all process same stream of data. Is there a variant of a Pipe that would let me have several equal readers?

Alternatively, if I do something like this, will pipe2 use the same segment (i.e. avoid needless copying of data)?

Memory<byte> memory = pipe1.Writer.GetMemory(10240);
int receivedBytesCount = await stream.ReadAsync(memory, token);

pipe1.Writer.Advance(receivedBytesCount);

await pipe2.Writer.WriteAsync(memory);
pipe2.Writer.Advance(receivedBytesCount);
Ivan Koshelev
  • 3,830
  • 2
  • 30
  • 50

1 Answers1

2

No, Pipes do not support multiple concurrent readers.

You might be able to build a "Tee" Pipe by:

  • Have a "writer" Pipe that reads from your source and writes to a Sequence
  • The Sequence would buffer the data to be read by a set of "reader" Pipes
  • Create multiple "reader" Pipes that read from the Sequence
  • Keep track of how far each reader has read and flush/release the processed data from the Sequence
  • Use backpressure from the readers to limit how much data you buffer into the Sequence. (i.e. Don't buffer more data in the sequence than you need)

This would be a non-trivial undertaking with plenty of places for things to go wrong. For example: you might need to place a lock around the Sequence as I'm not sure if it supports concurrent reading/writing.

Greg
  • 23,155
  • 11
  • 57
  • 79