Schema of codec is part of the data stream

Question

I'm currently evaluating if scodec is the right tool for my task. I have to parse an InputStream (file or network) which is structured the following:

| Header -  FieldDesc1 - FieldDesc2 - ...        \
- FieldDescM - Record1 - Record2 - ... - RecordN |

This means the stream starts with some metadata, which descibes what will follow. Each element is separated by a delimiter ( - ) which identifies what type it is. The N field descriptions contain the information which structure and size each of the N records will have.

I was readily able to parse header as well as the sequence of fields, because I was able to formulate a codec which is known at compile time. But I'm kind of puzzled how to build a codec at runtime due to the information from the field descriptions.

Is that possible? If yes, perhaps you can point me to an example?

Answer 1

Here is a starting point if it's still relevant:

• Use a DiscriminatorCodec (and a potentially related question), or
• Use consume() on the codec decoding the type identifier (which I guess is a simple number), and pass the type to a function returning the correct wanted Codec.

For example using consume(), you can determine what codec to use at decode time:

def variableTypeC = 
  int8.consume(tid => selectCodec(tid))(selectTypeId(_))

I had to work on a similar problem and went for the consume() solution (as I had the feeling it provided me with a bit more flexbility and was only discovering scodec at the time).

I'd be happy to build an example using DiscriminatorCodec if there is any need for it :).