3

I'm trying to write a small replacement for i3status, a small programm that comunicates with i3bar conforming this protocol. They exchange messeages via stdin and stdout.

The stream in both directions is an infinite array of json objects. The start of the stream from i3bar to i3status (which i want to replace) looks like this:

[
{"name": "some_name_1","instance": "some_inst_1","button": 1,"x": 213,"y": 35}
,{"name": "some_name_1","instance": "some_inst_2","button": 2,"x": 687,"y": 354}
,{"name": "some_name_2","instance": "some_inst","button": 1,"x": 786,"y": 637}
,{"name": "some_name_3","instance": "some_inst","button": 3,"x": 768,"y": 67}
...

This is an "array" of objects which represent clicks. The array will never close.

My question is now: What is the right way of parsing this?

Obviously I cannot use the json library because this is not a vaild json object.

Y123
  • 915
  • 13
  • 30
Tristan Storch
  • 690
  • 8
  • 18
  • There's bound to be a better way, but the easiest is probably just remove the `,` from the beginning, and then use the json library to parse the rest of the line: that is just a json object, right? (and the better way was probably posted while leaving this comment, so use that instead ;) that's why this is not an answer :) ) – Nanne Apr 02 '15 at 20:05
  • Exactly. The rest is just an json object. I can't do such hacky stuff. I would hate myself ;) – Tristan Storch Apr 02 '15 at 20:39
  • Strictly speaking, this is not valid JSON. A valid JSON array must have a terminating right square bracket (`]`), meaning it cannot be infinite. I would not expect a JSON parsing library to be able to handle it via the normal means, though you might be able to invoke some of its parsing functionality if it exposes it. – jpmc26 May 08 '19 at 21:51

3 Answers3

6

Write a custom reader function (or Decoder) which does a "streaming array parse" like so:

  1. Read and discard leading whitespace.
  2. If the next character is not a [ then return an error (can't be an array).
  3. While true do:
    1. Call json.Decoder.Decode into the "next" item.
    2. Yield or process the "next" item.
    3. Read and discard whitespace.
    4. If the next character is:
      1. A comma , then continue the for-loop in #3.
      2. A close bracket ] then exit the for-loop in #3.
      3. Otherwise return an error (invalid array syntax).
maerics
  • 151,642
  • 46
  • 269
  • 291
  • 1
    A basic state machine is the way to go. – sberry Apr 03 '15 at 06:20
  • How do I get the "next" item? Should I just use a scanner and split with the split function on newlines? – Tristan Storch Apr 03 '15 at 20:09
  • @TristanStorch: just call [`json.Decoder.Decode(...)`](https://golang.org/pkg/encoding/json/#Decoder.Decode) - from the docs, it "reads the next JSON-encoded value from its input" and should properly handle any irrelevant whitespace. – maerics Apr 04 '15 at 15:48
  • For the record: I solved it with [this](http://stackoverflow.com/questions/17676367/filtering-non-json-content-in-a-json-stream-in-go) approach. There is a tricky bit with json.Decoder. – Tristan Storch Apr 07 '15 at 15:34
4

I'm writing my own handler for click events in i3 as well. That's how I stumbled upon this thread.

The Golang standard library does actually do exactly what is required (Golang 1.12). Not sure it did when you asked the question or not?

// json parser read from stdin
decoder := json.NewDecoder(os.Stdin)

// Get he first token - it should be a '['
tk, err := decoder.Token()
if err != nil {
    fmt.Fprintln(os.Stderr, "couldn't read from stdin")
    return
}
// according to the docs a json.Delim is one [, ], { or }
// Make sure the token is a delim
delim, ok := tk.(json.Delim)
if !ok {
    fmt.Fprintln(os.Stderr, "first token not a delim")
    return
}
// Make sure the value of the delim is '['
if delim != json.Delim('[') {
    fmt.Fprintln(os.Stderr, "first token not a [")
    return
}

// The parser took the [ above
// therefore decoder.More() will return
// true until a closing ] is seen - i.e never 
for decoder.More() {
    // make a Click struct with the relevant json structure tags
    click := &Click{}

    // This will block until we have a complete JSON object
    // on stdin
    err = decoder.Decode(click)
    if err != nil {
            fmt.Fprintln(os.Stderr, "couldn't decode click")
            return
    }
    // Do something with click event
}
James Welchman
  • 105
  • 1
  • 6
0

What you are looking for is a Streaming API for JSON. There are many available a quick Google search revealed this project that does list Streaming as one of it's advance features.

Y123
  • 915
  • 13
  • 30
  • Thanks for the quick answer. I googled now myself for Go JSON Streaming APIs, but sadly the main part is just about sending (as your link). The receiving is all about newline delimited lists (`{...}\n{...}\n{...}...`) and no infinite array. For this I would have just used Scanners and json unmarshaling, which seams not to be the idiomatic way in my case. – Tristan Storch Apr 02 '15 at 20:37
  • I am sorry to hear that. Let's assume you have no means of controlling this infinite input as it will come in as an array and not as splits like you need. You can code a "pre-processor" to clean up before you ignoring the start and end box bracket is relatively trivial. Now to ignore the comma consider this create a counter with value 1, for every start { increment a counter and for every } decrement a counter - when the counter is 1 - ignore the incoming comma (and replace with a \n as needed by the API). – Y123 Apr 02 '15 at 21:36