4

I am making a streaming API which handles both RPC style calls as well as notifications from the server to the client (not the notifications in the JSON-RPC spec which are client to server). This last part unfortunately rules out JSON-RPC + persistent HTTP.

The API is based on the JSON and JSON-RPC specs.

JSON - http://www.ietf.org/rfc/rfc4627.txt

JSON-RPC - http://www.jsonrpc.org/specification

Typical session might be:

-> Sending to server
<- Receiving from server    

-> {'id': 0, 'method':'login','params':{'token':'secret'}}
<- {'id': 0, 'method':'login','result':0}
-> {'id': 1, 'method':'subscribe','params':{'symbol':'VOD.L'}}
<- {'id': 1, 'method':'subscribe','result':0}
...
<- {'method':'notifyPrice','params':{'symbol':'VOD.L', 'bid':10.1, 'ask':10.03}}
<- {'method':'notifyPrice','params':{'symbol':'VOD.L', 'bid':10.2, 'ask':10.03}}

The above messages, particularly the notifications, could come in any order and in the same packet. Neither of the specs seem to include details of a message separator which makes it difficult to know when an entire JSON message has been received without using a SAX based JSON parser, which are rather rare compared to their DOM counterparts.

Am I missing something obvious, or is there genuinely no standard way to separate between multiple JSON messages coming in over the wire?

jmc
  • 813
  • 10
  • 18

1 Answers1

3

You are missing something :-)

Each JSON-RPC message is a valid JSON value. A JSON value can by any of:

  • an Array
  • an Object
  • a String
  • a Number

The JSON-RPC envelope is an Object.

If this were a raw socket (like Telnet), then...

Objects start with { and end with }. If you're using the right kind of parser (pull, or event-oriented) at your receiver then it doesn't matter how nested the Objects and Arrays get, you'll still know when you hit that last }.

The above messages, particularly the notifications, could come in any order and in the same packet.

As long as the requests are not interlaved (one finishes before the next starts), then there's no problem at all. It's up to you whether you also require newline termination of the envelope (a.k.a line-oriented protocol).

However, as you're dealing with HTTP...

Why not just use batch messages per the JSON-RPC spec?

David-SkyMesh
  • 5,041
  • 1
  • 31
  • 38
  • Yes, I agree with everything you say, but as I said in my question, there are not many SAX/event orientated JSON parsers - the ones which their are not well tested / used. And also, I don't need an SAX/event orientated implementation - the messages are not long and I'm not short on memory and I'd prefer not to have to parse a message just to find when it terminates. Also, I'm not using HTTP and regardless using batching doesn't help as you still don't know when the batch ends without using a SAX/event-orientated parser. – jmc Jul 29 '14 at 09:19
  • Also, the question was "Is there genuinely no standard way to separate between multiple JSON messages coming in over the wire?" You've said I could use a newline terminator, but that's not in the spec and would surely cause incompatibilities with other JSON RPC libs? – jmc Jul 29 '14 at 09:22
  • `I could use a newline terminator, but that's not in the spec` In what spec? The JSON-RPC specification is transport-protocol agnostic. HTTP contains plently of newlines (which mean something to HTTP but not to JSON-RPC), and it's the most used transport protocol for JSON-RPC. The JSON-RPC spec deals with specifically structured JSON messages. It doesn't care what other protocol you embed those JSON messages inside. Did you take some time to read about `batch` messages? – David-SkyMesh Jul 30 '14 at 04:23
  • I think you're very confused about what the JSON-RPC spec says, what transport protocols are, and what various libraries do. Batching of messages occurs inside the JSON-RPC messages themselves. There's no need for you to do any additional parsing (other than recognising the end of the JSON-RPC message itself). If you're using a transport protocol that doesn't have a natural separator (i.e: raw TCP sockets) then you *MUST* track the protocol state yourself. And for that matter, SAX/event parsers are typically constant-memory parsers... why would memory be a concern? – David-SkyMesh Jul 30 '14 at 04:25
  • Ok, point taken about the separation between JSON-RPC and an underlying protocol. I'd missed that. Which is really the answer to the question - there's no way to separate the messages as that's not the responsibility of the JSON-RPC protocol. Good. :) – jmc Jul 30 '14 at 12:25
  • I did look at batch messages (for all it's detailed in the spec), hence my response `using batching doesn't help as you still don't know when the batch (or an individual message for that matter) ends without using a SAX/event-orientated parser`. If that's incorrect then perhaps tell me why? Memory isn't a concern (I said so), hence I would not want to use a SAX/event parser (to benefit from the constant memory usage) - the messages are processed elsewhere, processing them just to find their end is wasteful. – jmc Jul 30 '14 at 12:33
  • Ah, I misread the "short on memory" part. You've correctly understood the need for *some* sort of parser when you're dealing with multiple messages in a single transaction of the underlying transport protocol. – David-SkyMesh Jul 30 '14 at 12:40