Parse delimited and nested field names from URL parameter for partial response

Question

In a Flask-RESTful based API, I want to allow clients to retrieve a JSON response partially, via the ?fields=... parameter. It lists field names (keys of the JSON object) that will be used to construct a partial representation of the larger original.

This may be, in its simplest form, a comma-separated list:

GET /v1/foobar?fields=name,id,date

That can be done with webargs' DelimitedList schema field easily, and is no trouble for me.

---

But, to allow nested objects' keys to be represented, the delimited field list may include arbitrarily nested keys enclosed in matching parentheses:

GET /v1/foobar?fields=name,id,another(name,id),date
{
  "name": "",
  "id": "",
  "another": {
    "name": "",
    "id": ""
  },
  "date": ""
}

GET /v1/foobar?fields=id,one(id,two(id,three(id),date)),date
{
  "id": "",
  "one": {
    "id: "",
    "two": {
      "id": "",
      "three": {
        "id": ""
      },
      "date": ""
    }
  },
  "date": ""
}

GET /v1/foobar?fields=just(me)
{
  "just": {
    "me: ""
  }
}

---

My question is two-fold:

1. Is there a way to do this (validate & deserialize) with webargs and marshmallow natively?

2. If not, how would I do this with a parsing framework like pyparsing? Any hint on what the BNF grammar is supposed to look like is highly appreciated.

Answer 1

Pyparsing has a couple of helpful built-ins, delimitedList and nestedExpr. Here is an annotated snippet that builds up a parser for your values. (I also included an example where your list elements might be more than just simple alphabetic words):

import pyparsing as pp

# placeholder element that will be used recursively
item = pp.Forward()

# your basic item type - expand as needed to include other characters or types
word = pp.Word(pp.alphas + '_')
list_element = word

# for instance, add support for numeric values
list_element = word | pp.pyparsing_common.number

# retain x(y, z, etc.) groupings using Group
grouped_item = pp.Group(word + pp.nestedExpr(content=pp.delimitedList(item)))

# define content for placeholder; must use '<<=' operator here, not '='
item <<= grouped_item | list_element

# create parser
parser = pp.Suppress("GET /v1/foobar?fields=") + pp.delimitedList(item)

You can test any pyparsing expression using runTests:

parser.runTests("""
    GET /v1/foobar?fields=name,id,date
    GET /v1/foobar?fields=name,id,another(name,id),date
    GET /v1/foobar?fields=id,one(id,two(id,three(id),date)),date
    GET /v1/foobar?fields=just(me)
    GET /v1/foobar?fields=numbers(1,2,3.7,-26e10)    
    """,  fullDump=False)

Gives:

GET /v1/foobar?fields=name,id,date
['name', 'id', 'date']
GET /v1/foobar?fields=name,id,another(name,id),date
['name', 'id', ['another', ['name', 'id']], 'date']
GET /v1/foobar?fields=id,one(id,two(id,three(id),date)),date
['id', ['one', ['id', ['two', ['id', ['three', ['id']], 'date']]]], 'date']
GET /v1/foobar?fields=just(me)
[['just', ['me']]]
GET /v1/foobar?fields=numbers(1,2,3.7,-26e10)
[['numbers', [1, 2, 3.7, -260000000000.0]]]