Lazily read and manipulate float from stdin in haskell

Question

I'm trying to read data as Doubles from stdin, manipulate them and write them as well. What I've come up with so far is:

import qualified Data.ByteString.Lazy as B
import Data.Binary.IEEE754
import Data.Binary.Get

-- gives a list of doubles read from stdin
listOfFloat64le = do
  empty <- isEmpty
  if empty
     then return []
     else do v <- getFloat64le
             rest <- listOfFloat64le
             return (v : rest)


-- delay signal by one
delay us = 0 : us

-- feedback system, add delayed version of signal to signal
sys us = zipWith (+) us (delay us)

main = do
    input <- B.getContents
    let hs = sys $ runGet listOfFloat64le input
    print $ take 10 hs

The idea is to fead data to the program which is then passed through a feedback system before it is written to stdout. Although right now it just prints the first 10 values.

This works but does not seem to evaluate lazily. I.e it has to read all the input into memory. So:

dd if=/dev/urandom bs=8 count=10 | runhaskell feedback.hs

will work just fine but:

dd if=/dev/urandom | runhaskell feedback.hs

will not. My guess is it's the listOfFloat64le function that makes things not work properly. So how do I create an iterable to pass into my sys function without having to read everything into memory?

I'm not a very experienced haskeller.

Fulkerson · Answer 1 · 2012-09-20T16:29:13.027

1

I took another route by instead splitting the ByteString at intervals of 8 bytes and mapping over it instead:

import qualified Data.ByteString.Lazy as L
import Data.Binary.IEEE754
import Data.Binary.Get

-- delay signal by one
delay us = 0 : us

-- feedback system, add delayed version of signal to signal
sys us = zipWith (+) us (delay us)

-- split ByteString into chunks of size n
chunk n xs = if (L.null xs)
        then []
        else y1 : chunk n y2
          where
            (y1, y2) = L.splitAt n xs


main = do
    input <- L.getContents

    let signal = map (runGet getFloat64le) (chunk 8 input)
    print $ take 10 (sys signal)

This seem to work atleast but I don't know what the performance is like.

EDIT: I switched from chunk to chunker which uses runGetState instead:

chunker :: Get a -> L.ByteString -> [a]
chunker f input = if (L.null input)
                     then []
                     else val : chunker f rest
                       where
                        (val, rest, _) = runGetState f input 0

And using it like: let signal = chunker getFloat64le input

edited Sep 20 '12 at 16:29

answered Sep 20 '12 at 15:52

Fulkerson

118
2
8

If the input is arriving fast, and you consume a lot of it, the splitting will take some time. In that case, it may be better to entirely skip the `Get` monad and build a list of `Double`s from the input directly; read eight bytes from the `ByteString`, make a `Word64` from them, and convert that bit pattern to a `Double` (using e.g. [the last of these](http://hackage.haskell.org/packages/archive/data-binary-ieee754/0.4.2.1/doc/html/Data-Binary-IEEE754.html#g:3)). If the input is not arriving fast, the programme is IO-bound anyway. If you don't consume much, it doesn't matter anyway. – Daniel Fischer Sep 20 '12 at 16:06
I ended up modifying the chunk function to use `runGetState` instead. Is using the Get monad slower than plain reading and using wordToDouble or is it just splitting a ByteString that might be bad? – Fulkerson Sep 20 '12 at 16:30
Oh, "Deprecated: Use runGetPartial instead. This function will be removed." The implementation of `Get` has changed. I'm not sure whether ignoring the offset is okay for the old implementation. – Daniel Fischer Sep 20 '12 at 16:37
The `Get` monad brings in a little overhead, I don't think that'd be the big thing, though, I'd expect that the allocation needed for splitting the `ByteString` into so many short chunks would cost significantly more. But I haven't measured, so I might be wrong. – Daniel Fischer Sep 20 '12 at 16:40

score 0 · Answer 2 · edited May 23 '17 at 11:55

0

See this question. Looks like Binary become more strict than it was long time ago when I used it.

edited May 23 '17 at 11:55

Community

1
1

answered Sep 20 '12 at 14:45

ony

12,457
1
33
41

score 0 · Answer 3 · answered Sep 20 '12 at 17:07

0

This seems a standards problem where you can easily use something like pipes or conduits. You can make stdin as the source and stdout as the sink and apply the transformer as a conduit.

answered Sep 20 '12 at 17:07

Satvik

11,238
1
38
46

Lazily read and manipulate float from stdin in haskell

3 Answers3