7

I have a collection resource called Columns. A GET with Accept: application/json can't directly return a collection, so my representation needs to nest it in a property:-

{ "propertyName": [
   { "Id": "Column1", "Description": "Description 1" },
   { "Id": "Column2", "Description": "Description 2" }
  ]
}

Questions:

  1. what is the best name to use for the identifier propertyName above? should it be:

    • d (i.e. is d an established convention or is it specific to some particular frameworks (MS WCF and MS ASP.NET AJAX ?)
    • results (i.e. is results an established convention or is it specific to some particular specifications (MS OData)?)
    • Columns (i.e. the top level property should have a clear name and it helps to disambiguate my usage of generic application/json as the Media Type)

    NB I feel pretty comfortable that there should be something wrapping it, and as pointed out by @tuespetre, XML or any other representation would force you to wrap it to some degree anyway

  2. when PUTting the content back, should the same wrapping in said property be retained [given that it's not actually necessary for security reasons and perhaps conventional JSON usage idioms might be to drop such nesting for PUT and POST given that they're not necessary to guard against scripting attacks] ?

    my gut tells me it should be symmetric as for every other representation but there may be prior art for dropping the d/*results** [assuming that's the answer to part 1]*

    ... Or should a PUT-back (or POST) drop the need for a wrapping property and just go with:-

     [
           { "Id": "Column1", "Description": "Description 1" },
           { "Id": "Column2", "Description": "Description 2" }
     ]
    
    • Where would any root-level metadata go if one wished to add that?
    • How/would a person crafting a POST Just Know that it needs to be symmetric?

EDIT: I'm specifically interested in an answer that with a reasoned rationale that specifically takes into account the impacts on client usage with JSON. For example, HAL takes care to define a binding that makes sense for both target representations.

EDIT 2: Not accepted yet, why? The answers so far don't have citations or anything that makes them stand out over me doing a search and picking something out of the top 20 hits that seem reasonable. Am I just too picky? I guess I am (or more likely I just can't ask questions properly :D). Its a bit mad that a week and 3 days even with an )admittedly measly) bonus on still only gets 123 views (from which 3 answers ain't bad)

Ruben Bartelink
  • 59,778
  • 26
  • 187
  • 249
  • Unrelated, but still... it's "application/json", not "text/json" – Julian Reschke Sep 21 '12 at 12:52
  • @JulianReschke Thanks; fixed (BTW I'd have been happy for you to edit it in either) – Ruben Bartelink Sep 21 '12 at 14:13
  • If you return something like: while(true); {"propertyName": [ ... and remove the first line containing the infinite loop before you parse the rest as JSON, you'll be able to return a collection. – Localist Sep 25 '12 at 13:44
  • @Mohamed Not looking to do any crazy coding tricks or make up my own special non-standard. I'm pretty satisfied that wrapping in a property is an established convention for a JSON representation of a resource. – Ruben Bartelink Sep 25 '12 at 19:38

3 Answers3

3

Updated Answer

Addressing your questions (as opposed than going off on a bit of a tangent in my original answer :D), here's my opinions:

1) My main opinion on this is that I dislike d. As a client consuming the API I would find it confusing. What does it even stand for anyway? data?

The other options look good. Columns is nice because it mirrors back to the user what they requested.

If you are doing pagination, then another option might be something like page or slice as it makes it clear to the client, that they are not receiving the entire contents of the collection.

{
    "offset": 0,
    "limit": 100,
    "page" : [
        ...
    ]
}

2) TBH, I don't think it makes that much difference which way you go for this, however if it was me, I probably wouldn't bother sending back the envelope, as I don't think there is any need (see below) and why make the request structure any more complicated than it needs to be?

I think POSTing back the envelope would be odd. POST should let you add items into the collection, so why would the client need to post the envelope to do this?

PUTing the envelope back could make sense from a RESTful standpoint as it could be seen as updating metadata associated with the collection as a whole. I think it is worth thinking about the sort of meta data you will be exposing in the envelope. All the stuff I think would fit well in this envelope (like pagination, aggregations, search facets and similar meta data) is all read only, so it doesn't make sense for the client to send this back to the server. If you find yourself with a lot of data in the envelope that the client is able to mutate - then it could be a sign to break that data out into a separate resource with the list as a sub collection. Rubbish example:

/animals

{
    "farmName": "farm",
    "paging": {},
    "animals": [
        ...
    ]
}

Could be broken up into:

/farm/1

{
    "id": 1,
    "farmName": "farm"
}

and

/farm/1/animals

{
    "paging": {},
    "animals": [
        ...
    ]
}

Note: Even with this split, you could still return both combined as a single response using something like Facebook's or LinkedIn's field expansion syntax. E.g. http://example.com/api/farm/1?field=animals.offset(0).limit(10)

In response, to your question about how the client should know what the JSON payload they are POSTing and PUTing should look like - this should be reflected in your API documentation. I'm not sure if there is a better tool for this, but Swagger provides a spec that allows you to document what your request bodies should look like using JSON Schema - check out this page for how to define your schemas and this page for how to reference them as a parameter of type body. Unfortunately, Swagger doesn't visualise the request bodies in it's fancy web UI yet, but it's is open source, so you could always add something to do this.

Original Answer

Check out William's comment in the discussion thread on that page - he suggests a way to avoid the exploit altogether which means you can safely use a JSON array at the root of your response and then you need not worry about either of you questions.

The exploit you link to relies on your API using a Cookie to authenticate a user's session - just use a query string parameter instead and you remove the exploit. It's probably worth doing this anyway since using Cookies for authentication on an API isn't very RESTful - some of your clients may not be web browsers and may not want to deal with cookies.

Why Does this fix work?

The exploit is a form of CSRF attack which relies on the attacker being able to add a script tag on his/her own page to a sensitive resource on your API.

<script src="http://mysite.com/api/columns"></script> 

The victims web browser will send all Cookies stored under mysite.com to your server and to your servers this will look like a legitimate request - you will check the session_id cookie (or whatever your server-side framework calls the cookie) and see the user is authenticated. The request will look like this:

GET http://mysite.com/api/columns
Cookie: session_id=123456789;

If you change your API you ignore Cookies and use a session_id query string parameter instead, the attacker will have no way of tricking the victims web browser into sending the session_id to your API.

A valid request will now look like this:

GET http://mysite.com/api/columns?session_id=123456789

If using a JavaScript client to make the above request, you could get the session_id from a cookie. An attacker using JavaScript from another domain will not be able to do this, as you cannot get cookies for other domains (see here).

Now we have fixed the issue and are ignoring session_id cookies, the script tag on the attackers website will still send a similar request with a GET line like this:

GET http://mysite.com/api/columns

But your server will respond with a 403 Forbidden since the GET is missing the required session_id query string parameter.

What if I'm not authenticating users for this API?

If you are not authenticating users, then your data cannot be sensitive and anyone can call the URI. CSRF should be a non-issue since with no authentication, even if you prevent CSRF attacks, an attacker could just call your API server side to get your data and use it in anyway he/she wants.

Community
  • 1
  • 1
theon
  • 14,170
  • 5
  • 51
  • 74
  • +1 Thanks for a thorough response and taking the time to explain the pros and cons in depth. While I'm fully hypermedia and all that and could start emitting session-specific links, I'd like to reiterate that I have no beef with having to embed arrays in a property - and agree with @tuespetre that it can actually be a more understandable / Pit of Successy design. My question is centred around a) what do people do on the GET side (esp the naming and/or whether to lean on the a specific `application/json-?/` Media Type w/ JSON) and b) does the PUT/POST side ever vary wrt having a root node – Ruben Bartelink Sep 26 '12 at 21:47
  • Ah sure. I really should have read your comments from yesterday on the question. I also agree that wrapping arrays in an object can be a good idea - we are planning to do this on the REST API we're currently working on, and not just for arrays at the root of the document, but for all arrays throughout. Our main reason was to provide a place for pagination meta data for each array in the document. – theon Sep 26 '12 at 22:56
  • Good point re having somewhere for the metadata to live. I'm using HAL, and your point is relevant to that, though having `{_embedded { Tags: { Items: [ {"Id": 1, "Desc": "desc"}], _metadata1; 2 }}}` gets pretty ugly fast. I'm leaning by instinct towards a YAGNI there (having said that [in HAL] having an extra layer allows `_links` for the collection to have a home... Another point for people to consider in their answers (or to ask on the HAL mailing list) – Ruben Bartelink Sep 27 '12 at 08:07
  • At this instant, this answer is the answer I happen to have got the most from (the page metadata reasoning for example). Would be interested in your considered answers to the core questions 1) `d` or `Columns` 2) for PUT, `d` or `Columns` or nothing. I think it wouldnt be fair to award the bonus to this one without getting the question a bit more well ... answered – Ruben Bartelink Oct 01 '12 at 13:54
  • Thanks for the extra effort although I was hoping for more citations than opinion. That combined with the fact that I happened to get the most realisations from you. In terms of citations and meeting the terms of reference, I can't make this the accepted answer though. I'm a bit reluctant to buy into a 'the docs will make whatever choice you make for whatever reason' stance. On a tangent, can I recommend reading the HAL Spec (linked from this [intro blog post](http://blog.stateless.co/post/13296666138/json-linking-with-hal)) to you? – Ruben Bartelink Oct 02 '12 at 08:52
  • Thanks, I don't have much in the way of citations I'm afraid. I've read the HAL spec before, but we've have opted to try [JSON-LD](http://json-ld.org/) instead as it fit a bit better with some of our requirements. – theon Oct 02 '12 at 16:16
  • Interesting. It's no quick scan. I was turned off it by the fact that HAL certainly for now covers most of the bases for our stuff (and that [@mnot's opinionated linking write-up](http://www.mnot.net/blog/2011/11/25/linking_in_json) didnt make it interesting enough to do anything that scan. It'll be interesting to see how the space evolves over the cominy years... – Ruben Bartelink Oct 02 '12 at 19:44
  • Me too - it's all very up in the air at the moment and hard to tell if one will come out on top! Thanks for that link - interesting read. – theon Oct 02 '12 at 21:16
2
  1. I would go for 'd' because it clearly separates the 'envelope' of your resource from its content. This would also make it easier for consumers to parse your responses, as opposed to 'guessing' the name of the wrapping property of a given resource before being able to access what it holds.

  2. I think you're talking about two different things:

    • POST request should be sent in application/x-www-form-urlencoded. Your response should basically mirror a GET if you choose to include a representation of the newly created resource in your reply. (not mandatory in HTTP).
    • PUTs should definitely be symmetric to GETs. The purpose of a PUT request is to replace an existing resource representation with another. It just makes sense to have both requests share the same conventions, doesn't it?
Ruben Bartelink
  • 59,778
  • 26
  • 187
  • 249
Localist
  • 1,094
  • 1
  • 8
  • 14
  • +1 Thanks for thinking about it and answering. If one goes with the 'it's just an envelope' argument, am I correct in making the following additional inferences:- For PUT, wrap the collection in `d` because it should be symmetric. For POST, you feel I should break from the normal representation, even if e.g. I have a tree of properties in the GET - have you done that for an interesting resource - how do you map (e.g. multiple) nested properties?. Bonus Q - why is `d` a de facto standard - do you believe it has critical mass (it's not the most searchable and I'd like something to justify it) – Ruben Bartelink Sep 30 '12 at 01:51
  • Giving you right of reply before I assign the bonus - will delete this comment when you ack it and/or respond. – Ruben Bartelink Oct 01 '12 at 13:45
  • Unfortunately I didn't find the answer complete enough in terms of meeting the terms of reference of the question to pick this as the bonus'd one. But I agree with its current ranking as the top answer. – Ruben Bartelink Oct 02 '12 at 08:55
1
  1. Go with 'Columns' because it is semantically meaningful. It helps to think of how JSON and XML could mirror each other.

  2. If you would PUT the collection back, you might as well use the same media type (syntax, format, what you will call it.)

tuespetre
  • 1,827
  • 2
  • 18
  • 30
  • Thanks; this is my instinct too. Do you have any specific citations or examples to support the stance? – Ruben Bartelink Sep 22 '12 at 07:05
  • Edited in a clarification expanding on my comment a bit – Ruben Bartelink Sep 22 '12 at 07:17
  • It is you who defines the media types to be used in communicating between the client and server. You could do whatever you wanted to, honestly -- but it would be a lot easier to understand the format of your media type if it carried semantic meaning, and it would also be easier if you could re-use a media type as much as possible. That's where I gave my answer. This is a great blog post from Roy Fielding, read the third bullet point in particular: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven – tuespetre Sep 22 '12 at 14:04
  • +1 for the following up and a generally good post. The general advice is fine, but for json, the specifics are critical - e.g. for XML, its pretty natural to name one's elements ``. For me there is no mapping that is as obvious in JSON, and introducing redundant levels of hierarchy just to add back in all the meaning that has been 'lost' (when contrasted to XML) without namespaces and element names is for me against the spirit of JSON. Anyway, hopefully we'll get some more answers in due course... – Ruben Bartelink Sep 23 '12 at 01:24
  • Look at it this way: in order to return a single JS object that contains an array like you want, you have to have a property name for that array. Just like in XML, you have to have some 'root'. When you look at both JSON and XML, the structures are virtually identical -- it's just the syntax that differs. The 'spirit of JSON' as you put it is for JSON to be easily processed; you don't need some extra processing script or library to receive JSON from an asynchronous call. But data is data is data, and data has its structure regardless of the syntax you use. – tuespetre Sep 25 '12 at 00:41
  • Yes, makes sense. Still leaves unanswered whether people believe it's a better approach to use the Media Type to convey that and leave just `d` (I'm guessing no, but it's just that - guessing). And whether it being symmetric is trumped by not having to wrap a value one is PUTting or POSTing back in a property **in common practice** (as opposed to saying it should be `d` coz Ms AJAX can't be wrong!). (All this jabber doesn't actually mean I think you're necessarily wrong BTW) – Ruben Bartelink Sep 26 '12 at 19:27
  • I'm torn about whether this should be the answer to which the bonus is assigned - The reasoning is very solid, but some of the questions (should there be a all-but-redundant root node in what is PUT, and why?) are not fully closed. Not ruling it out for now. – Ruben Bartelink Oct 01 '12 at 13:49
  • It was a hard choice not to allocate the bonus to you as you have an answer that I don't disagree with and took the time to comment back, sorry. – Ruben Bartelink Oct 02 '12 at 08:57
  • I'm so hurt! ...just kidding. :) Hopefully this period of time hasn't held you back from coding, I know the conceptual/practices discussion and consideration can become an obsession. – tuespetre Oct 02 '12 at 14:20
  • Never! (It all just goes on the stack of things to eventually get around to as the system evolve - as it happens, in order to support the way things need to be to support [`$inlinecount` in OData](http://stackoverflow.com/questions/11300498/implementing-odata-inlinecount-using-asp-net-webapi/11319330#comment17137655_11319330) I'm now likely to end up using `results` as the property name – Ruben Bartelink Oct 02 '12 at 19:27