Questions tagged [haskell-streaming]
15 questions
3
votes
1 answer
Streaming pipeline using result in next step
I'm using the streaming package.
I want to use result of one step defined by the S.store as a parameter to a following step in the pipeline by preserving the constant memory. The myStream is loaded and parsed from a file.
I have a following example…
xbalaj
- 977
- 1
- 8
- 14
3
votes
1 answer
Intuition behind the functor in Stream f m r of the streaming library
A simplified (non-effectful) definition of Stream from the streaming library reads like this:
data Stream f = Step (f (Stream f))
| Return
I'm trying to understand what the motivation for introducing this functor f is.
A typical such…
mcmayer
- 1,931
- 12
- 22
3
votes
1 answer
Handling sum encoding in Streaming libraries
The motivation behind this question is this scenario - we have a stream of values which are represented by a Sum encoding. Let us assume Either ByteString ByteString where we represent streams of bytes in error and good states respectively. Now, we…
Sal
- 4,312
- 1
- 17
- 26
2
votes
1 answer
Generalize the merge function of the Haskell "Streaming" library
The goal is to generalize the Streaming.merge function,
merge :: (Monad m, Ord a) => Stream (Of a) m r -> Stream (Of a) m s -> Stream (Of a) m (r, s)
to an arbitrary number of source streams. The strategy is to use a Data.Heap.Heap of Stream (Of…
mcmayer
- 1,931
- 12
- 22
2
votes
1 answer
Why is streaming-bytestring giving me error "openBinaryFile: resource exhausted (Too many open files)"?
The streaming-bytestring library gives an error after printing about 512 bytes.
Error:
openBinaryFile: resource exhausted (Too many open files)
Code:
import Control.Monad.Trans (lift, MonadIO)
import …
paperduck
- 1,175
- 1
- 16
- 26
2
votes
2 answers
Streaming bytestring as WAI HTTP server response body
I have a value body :: BS.ByteString (ResourceT IO) (), from a function based on BS.readFile. I want to stream that value as the response body from a Wai Application. There's a helper, streamingResponse that takes a value of the type Stream (Of…
Matt Joiner
- 112,946
- 110
- 377
- 526
1
vote
1 answer
Haskell streaming - how to merge original stream with result stream
Using Haskell-streaming, I can easily group a stream and take sum on each group.
>>> S.print $ mapped S.toList $ S.groupBy (\ x y -> x*y>0) $ each [-1,-2,3,4,5,-6]
[-1,-2]
[3,4,5]
[-6]
>>> S.print $S.map sum $ mapped S.toList $ S.groupBy (\ x y ->…
Kevin Zhu
- 2,746
- 26
- 23
1
vote
1 answer
Haskell streaming - how to separate 1 stream into 2 after copy?
In haskell streaming, there is an example of copy
>>> (S.toList . mapped S.toList . chunksOf 5) $ (S.toList . mapped S.toList . chunksOf 3) $ S.copy $ each [1..10]
[[1,2,3,4,5],[6,7,8,9,10]] :> ([[1,2,3],[4,5,6],[7,8,9],[10]] :> ())
Is it…
Kevin Zhu
- 2,746
- 26
- 23
1
vote
2 answers
Missing type of result? In function length :: Monad m => ByteString m r -> m (Of Int r)
I have a simple function that reads a binary file one byte at a time. It get a compile time error, below. The problem seems to be that bs2, the resulting ByteString of BSSC.length, has an unknown type. Am I missing a type constraint on r?
import…
paperduck
- 1,175
- 1
- 16
- 26
1
vote
1 answer
How to access Data.Functor.Of?
I am trying to use the length function of the streaming-bytestring Data.ByteString.Streaming.Char8 library.
I see that the return value has type Of, but I am not clear on how to examine it. I tried using case, but the compiler says Not in scope:…
paperduck
- 1,175
- 1
- 16
- 26
1
vote
1 answer
Adapting Store decoding for streaming library
I am trying to adapt Store encoding and decoding for Streaming. Store already implements a streaming decode with a function called decodeMessageBS.
I tried to do a basic implementation of store deserialization for Streaming as below (without…
Sal
- 4,312
- 1
- 17
- 26
1
vote
1 answer
How to track progress through a streaming ByteString?
I'm using the streaming-utils streaming-utils to stream a HTTP response body. I want to track the progress similar to how bytestring-progress allows with lazy ByteStrings. I suspect something like toChunks would be necessary, then reducing some…
Matt Joiner
- 112,946
- 110
- 377
- 526
0
votes
1 answer
Combining ResourceT with bracket in a streaming pipeline
Here a simplification of my code:
import Database.PostgreSQL.Simple (Connection)
import qualified Streaming.Prelude as S
import Streaming.ByteString.Char8 as C
import Streaming.Zip (gunzip)
import Streaming
main :: IO ()
main = do
res <-…
xbalaj
- 977
- 1
- 8
- 14
0
votes
1 answer
When to call runResourceT on streaming-bytestring?
I am a Haskell beginner and still learning about monad transformers.
I am trying to use the streaming-bytestring library to read a binary file, process chunks of bytes, and print the result as each chunk is processed. I believe this is the popular…
paperduck
- 1,175
- 1
- 16
- 26
0
votes
1 answer
Append a delay to each chunk with the streaming library?
newbie to Streaming and Haskell here.
I've been playing around with the streaming library and I'm particularly interested in understanding the chunks part. Eg:
S.print $ S.delay 1.0 $ concats $ chunksOf 2 $ S.each [1..10]
Or:
S.print $ concats $…
Adam Schneider
- 3
- 1