HTML ─ Python dictionary from form-subelement via CherryPy

Question

# ! /usr/bin/env python
# -*- coding: utf-8 -*-
# image_slide.py

""" Python        2.7.3
    Cherrypy      3.2.2
"""

When an HTML form is submitted to a Python function, the values are sent within one dictionary. For some subelements in an HTML form, I want to POST an additional nested dictionary, within that form dictionary. The question is, if there is an HTML tag or such a thing, which Python would interpret as opening for a new dictionary. So far I tried naming the <td> or an invisible <fieldset>, but of course it isn't as simple. And nested forms aren't possible either.

What I have is a form, which contains a few text inputs and a table with a selection of associated items.

_____ the entire form __________________________________________________________
|                                                                              |
|   beverage:      coffee                                                      |
|   time of day:   morning                                                     |
|   temperature:   hot                                                         |
|   color:         brown                                                       |
|   taste:         full-bodied                                                 |
|                                                                              |
|   Ingredients:                                                               |
|   ________________________________________________________________________   |
|   |                   |                   |                   |          |   |
|   |     <IMAGE_1>     |     <IMAGE_2>     |     <IMAGE_3>     |     <IMAG|   |
|   |                   |                   |                   |          |   |
|   | filename: coffee  | filename: sugar   | filename: milk    | filename:|   |
|   | x select item     | x select item     | x select item     | x select |   |
|   |___________________|___________________|___________________|__________|   |
|   |_____________________________<scrollbar>______________________________|   |
|                                                                              |
|   < OK >                                                                     |
|______________________________________________________________________________|

Following are three simple pseudo-code snippets to explain the gist.

• Content of one form entry:

  beverages = {
      'coffee': {
          'beverage':    'coffee',
          'time of day': 'morning',
          'temperature': 'hot',
          'color':       'brown',
          'taste':       'full-bodied',
          }
      }
  
  ingredients = {
      'coffee': [
          'coffee',
          'sugar',
          'milk',
          'cinnamon',
          ],
      }
  

• Top part of the form :

  yield(u'''
      <form action="..." method="POST">
      <table>
  ''')
  
  for key, value in beverages['coffee'].items():
      yield(u'''
          <tr>
          <td> {key}: </td>
          <td> <input type="text" name="{key}" value="{value}">< /td>
          </tr>
          '''.format(locals(),)
  

• Bottom part of the form :

  """ This hole bottom part already shall be in a dictionary named 'ingredients'.
      For each loop pass, I'd like a sub-dictionary, named by its loop pass.
      I used the fictional tag <SPECIAL_SUB_DICT>, to open a new dictionary.
  """
  
  yield(u'''
      </table>
      <SPECIAL_SUB_DICT name="ingredients">
      <table style="overflow-x: scroll">
      <tr>
  ''')
  
  for index, name in enumerate(ingredients['coffee']):
      yield(u'''
          <td>
          <SPECIAL_SUB_DICT name="{index}">
          <img src="...">
          <input type="text" name="filename" value="{name}"> 
          <input type="check" name="chosen_one" value="{index}"> select item
          </SPECIAL_SUB_DICT>
          </td>
          '''.format(locals(),)
  
  yield(u'''
      </tr>
      </table>
      </SPECIAL_SUB_DICT>
      <input type="submit" name="submit" value="OK">
      </form>
      ''')
  

Conclusion

The problem is, without some sort of <SPECIAL_SUB_DICT> tag, I can't easily rename single ingredients. For example, if I'd like to change the filename from 'milk' to 'hole cream milk'. The way I do it now is, to add the current loop pass to the input name, like this:

'<input type="text" name="filename_{index}" value="{name}"> '.format(locals(),)

Then, within the receiving function, I can check which key starts with 'filename_' and update all of them:

for key, values in kwargs:
    if key startswith('filename_'):
        ingredients['coffee'][key[9:]] = value

It would be much nicer, if I just could iterate kwargs['ingredients']:

for key, values in kwargs['ingredients'].items():
    ingredients['coffee'][key] = value[filename]

I'm asking, because a <SPECIAL_SUB_DICT> tag would be much closer to my current solution than parsing the table with Python's BeautifulSoup. And of course I want to know it. After all, with BeautifulSoup I'd probably be done by now.

EDIT 1:

This runs with the web application framework CherryPy.
Maybe there's a way it could handle such a request.
Though I dont't think, that it would offer something off-standard.

EDIT 2:

Given, that the form-dictionary starts in the URL with a question mark, I don't think a sub-dictionary is possible, because I don't know of any dictionary closing character. The closest thing I could come up with is using a hidden input named 'index', and zip() it with the text input named 'filename'. This answer, unfortunately for PHP, got me in the right direction.

'<input type="hidden" name="index" value="{index}"> '.format(locals(),)
'<input type="text" name="filename" value="{name}"> '.format(locals(),)

for key, values in zip(kwargs['index'], kwargs['filename']):
    ingredients['coffee'][key] = value

However, this wouldn't work for the delete button, which I planed to add to each ingredient. Since only the submit input can carry the index, and its value is used to display the button's unicode symbol, I again have to append the index to the name.

Then, the only thing I can think of, is a form for each ingredient and the top part of the formular, instead of one large form for everything. Though I don't know if that would be good for the performance, if the list of ingredients becomes too long.

As a visual trick comes a background image to mind, to replace the value, which then transparently, can be used according to its name. This answer and its corresponding question was helpful to get it working.

But nothing of this is an answer to my question.
I'm still missing the one solution for all inputs,
like multiple forms, just more elegant.

Answer 1

Okay, I wanted to know how this is done as well, so here you go. This only works for single-level dicts. If you want to make this recursive, I'll leave it to you.

The problem with this: <input name="foo[bar]"> is that foo[bar] is the variable name, and the brackets are converted to URL-safe entities:

[DEBUG] BODY_TEXT: foo%5Bbar%5D=input_value

CherryPy allows you to define a custom body processor by defining a callable in cherrypy.request.body.processors, which is a dictionary whose keys match the content type. This is how the cherrypy.tools.json_in() tool works, by replacing the body processor with a function that parses JSON when the request's content type is application/json.

The final part you need is how to inject your parsed dictionary (i.e. {'foo': 'bar'} into the handler method's arguments to be used just like regular POST input vars. I took it directly from the default form processor: https://github.com/cherrypy/cherrypy/blob/main/cherrypy/_cpreqbody.py#L177

So here's the tool. You need to enable it in your config, and load it in your server script, but it will take a dict-shaped set of form inputs pass the dict to your handler. It does this with a regular expression.

import re
import cherrypy
import logging
import urllib

class ImprovedFormInput(cherrypy.Tool):

    def __init__(self):
        logging.debug('ImprovedFormInput.__init__()')

        cherrypy.Tool.__init__(self, 'before_request_body', self.handle_post, priority=50)

    def handle_post(self):
        logging.debug('ImprovedFormInput.handle_post()')

        request = cherrypy.serving.request

        def impfin_processor(entity):

            raw_body = entity.fp.read()

            body_text = raw_body.decode()
            logging.debug('BODY_TEXT: %s', body_text)

            parsed = urllib.parse.parse_qs(body_text)
            logging.debug('PARSED: %s', parsed.keys())

            form_inputs = {}
            r = re.compile(r'^(\w+)\[(.+)\]$') # This pattern could be better

            for key in parsed.keys():

                m = r.match(key)
                if m is None:
                    continue

                groups = m.groups()

                pkey = groups[0]
                logging.debug('PKEY: %s', pkey)

                ckey = groups[1]
                logging.debug('CKEY: %s', ckey)

                if pkey not in form_inputs:
                    form_inputs[pkey] = {}

                form_inputs[pkey][ckey] = parsed[key]

            logging.debug('FINPUTS: %s', form_inputs)

            # https://github.com/cherrypy/cherrypy/blob/main/cherrypy/_cpreqbody.py#L177
            for key, value in form_inputs.items():
                if key in entity.params:
                    if not isinstance(entity.params[key], list):
                        entity.params[key] = [entity.params[key]]
                    entity.params[key].append(value)
                else:
                    entity.params[key] = value

        request.body.processors['application/x-www-form-urlencoded'] = impfin_processor