type=dict in argparse.add_argument()

Question

I want to use the standard library argparse module to parse command line arguments to my program, and have the program accept an optional argument -i (or --image) which is a dictionary.

I tried configuring the parser like this:

parser.add_argument('-i','--image', type=dict, help='Generate an image map from the input file (syntax: {\'name\': <name>, \'voids\': \'#08080808\', \'0\': \'#00ff00ff\', \'100%%\': \'#ff00ff00\'}).')

But when I try to run the script, I get an error:

$ ./script.py -i {'name': 'img.png','voids': '#00ff00ff','0': '#ff00ff00','100%': '#f80654ff'}
    
script.py: error: argument -i/--image: invalid dict value: '{name:'

Even though similar syntax would work fine inside the interpreter:

>>> a={'name': 'img.png','voids': '#00ff00ff','0': '#ff00ff00','100%': '#f80654ff'}

How should I a) write the command line and b) set up the argparse logic?

Answer 1

Necroing this: json.loads works here, too. It doesn't seem too dirty.

import json
import argparse

test = '{"name": "img.png","voids": "#00ff00ff","0": "#ff00ff00","100%": "#f80654ff"}'

parser = argparse.ArgumentParser()
parser.add_argument('-i', '--input', type=json.loads)

args = parser.parse_args(['-i', test])

print(args.input)

Returns:

{u'0': u'#ff00ff00', u'100%': u'#f80654ff', u'voids': u'#00ff00ff', u'name': u'img.png'}

Answer 2

For completeness, and similarly to json.loads, you could use yaml.load (available from PyYAML in PyPI). This has the advantage over json in that there is no need to quote individual keys and values on the command line unless you are trying to, say, force integers into strings or otherwise overcome yaml conversion semantics. But obviously the whole string will need quoting as it contains spaces!

>>> import argparse
>>> import yaml
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('-fna', '--filename-arguments', type=yaml.load)
>>> data = "{location: warehouse A, site: Gloucester Business Village}"
>>> ans = parser.parse_args(['-fna', data])
>>> print ans.filename_arguments['site']
Gloucester Business Village

Although admittedly in the question given, many of the keys and values would have to be quoted or rephrased to prevent yaml from barfing. Using the following data seems to work quite nicely, if you need numeric rather than string values:

>>> parser.add_argument('-i', '--image', type=yaml.load)
>>> data = "{name: img.png, voids: 0x00ff00ff, '0%': 0xff00ff00, '100%': 0xf80654ff}"
>>> ans = parser.parse_args(['-i', data])
>>> print ans.image
{'100%': 4161164543L, 'voids': 16711935, 'name': 'img.png', '0%': 4278255360L}

Answer 3

Using simple lambda parsing is quite flexible:

parser.add_argument(
    '--fieldMap',
    type=lambda x: {k:int(v) for k,v in (i.split(':') for i in x.split(','))},
    help='comma-separated field:position pairs, e.g. Date:0,Amount:2,Payee:5,Memo:9'
)

Answer 4

I’ll bet your shell is messing with the braces, since curly braces are the syntax used for brace expansion features in many shells (see here).

Passing in a complex container such as a dictionary, requiring the user to know Python syntax, seems a bad design choice in a command line interface. Instead, I’d recommend just passing options in one-by-one in the CLI within an argument group, and then build the dict programmatically from the parsed group.

Answer 5

Combining the type= piece from @Edd and the ast.literal_eval piece from @Bradley yields the most direct solution, IMO. It allows direct retrieval of the argval and even takes a (quoted) default value for the dict:

Code snippet

parser.add_argument('--params', '--p', help='dict of params ', type=ast.literal_eval, default="{'name': 'adam'}")
args = parser.parse_args()

Running the Code

python test.py --p "{'town': 'union'}"

note the quotes on the dict value. This quoting works on Windows and Linux (tested with [t]csh).

Retrieving the Argval

dict=args.params

Answer 6

You can definitely get in something that looks like a dictionary literal into the argument parser, but you've got to quote it so when the shell parses your command line, it comes in as

• a single argument instead of many (the space character is the normal argument delimiter)
• properly quoted (the shell removes quotes during parsing, because it's using them for grouping)

So something like this can get the text you wanted into your program:

python MYSCRIPT.py -i "{\"name\": \"img.png\", \"voids\": \"#00ff00ff\",\"0\": \"#ff00ff00\",\"100%\": \"#f80654ff\"}"

However, this string is not a valid argument to the dict constructor; instead, it's a valid python code snippet. You could tell your argument parser that the "type" of this argument is eval, and that will work:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('-i','--image', type=eval, help='Generate an image map...')
args = parser.parse_args()
print args

and calling it:

% python MYSCRIPT.py -i "{\"name\": \"img.png\", \"voids\": \"#00ff00ff\",\"0\": \"#ff00ff00\",\"100%\": \"#f80654ff\"}"
Namespace(image={'0': '#ff00ff00', '100%': '#f80654ff', 'voids': '#00ff00ff', 'name': 'img.png'})

But this is not safe; the input could be anything, and you're evaluating arbitrary code. It would be equally unwieldy, but the following would be much safer:

import argparse
import ast

parser = argparse.ArgumentParser()
parser.add_argument('-i','--image', type=ast.literal_eval, help='Generate an image map...')
args = parser.parse_args()
print args

This also works, but is MUCH more restrictive on what it will allow to be eval'd.

Still, it's very unwieldy to have the user type out something, properly quoted, that looks like a python dictionary on the command line. And, you'd have to do some checking after the fact to make sure they passed in a dictionary instead of something else eval-able, and had the right keys in it. Much easier to use if:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--image-name", required=True)
parser.add_argument("--void-color", required=True)
parser.add_argument("--zero-color", required=True)
parser.add_argument("--full-color", required=True)

args = parser.parse_args()

image = {
    "name": args.image_name,
    "voids": args.void_color,
    "0%": args.zero_color,
    "100%": args.full_color
    }
print image

For:

% python MYSCRIPT.py --image-name img.png --void-color \#00ff00ff --zero-color \#ff00ff00 --full-color \#f80654ff
{'100%': '#f80654ff', 'voids': '#00ff00ff', 'name': 'img.png', '0%': '#ff00ff00'}

Answer 7

One of the simplest ways I've found is to parse the dictionary as a list, and then convert that to a dictionary. For example using Python3:

#!/usr/bin/env python3
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('-i', '--image', type=str, nargs='+')
args = parser.parse_args()
if args.image is not None:
    i = iter(args.image)
    args.image = dict(zip(i, i))
print(args)

then you can type on the command line something like:

./script.py -i name img.png voids '#00ff00ff' 0 '#ff00ff00' '100%' '#f80654ff'

to get the desired result:

Namespace(image={'name': 'img.png', '0': '#ff00ff00', 'voids': '#00ff00ff', '100%': '#f80654ff'})

Answer 8

General Advice: DO NOT USE eval.

If you really have to ... "eval" is dangerous. Use it if you are sure no one will knowingly input malicious input. Even then there can be disadvantages. I have covered one bad example.

Using eval instead of json.loads has some advantages as well though. A dict doesn't really need to be a valid json. Hence, eval can be pretty lenient in accepting "dictionaries". We can take care of the "danger" part by making sure that final result is indeed a python dictionary.

import json
import argparse

tests = [
  '{"name": "img.png","voids": "#00ff00ff","0": "#ff00ff00","100%": "#f80654ff"}',
  '{"a": 1}',
  "{'b':1}",
  "{'$abc': '$123'}",
  '{"a": "a" "b"}' # Bad dictionary but still accepted by eval
]
def eval_json(x):
  dicti = eval(x)
  assert isinstance(dicti, dict)
  return dicti

parser = argparse.ArgumentParser()
parser.add_argument('-i', '--input', type=eval_json)
for test in tests:
  args = parser.parse_args(['-i', test])
  print(args)

Output:

Namespace(input={'name': 'img.png', '0': '#ff00ff00', '100%': '#f80654ff', 'voids': '#00ff00ff'})
Namespace(input={'a': 1})
Namespace(input={'b': 1})
Namespace(input={'$abc': '$123'})
Namespace(input={'a': 'ab'})

Answer 9

A minimal example to pass arguments as a dictionary from the command line:

# file.py
import argparse
import json
parser = argparse.ArgumentParser()
parser.add_argument("-par", "--parameters",
                    required=False,
                    default=None,
                    type=json.loads
                )
args = parser.parse_args()
print(args.parameters)

and in the terminal you can pass your arguments as a dictionary using a string format:

python file.py --parameters '{"a":1}'

Answer 10

Here is a another solution since I had to do something similar myself. I use the ast module to convert the dictionary, which is input to the terminal as a string, to a dict. It is very simple.

Code snippet

Say the following is called test.py:

import argparse
import ast

parser = argparse.ArgumentParser()
parser.add_argument('--params', '--p', help='dict of params ',type=str)

options = parser.parse_args()

my_dict = options.params
my_dict = ast.literal_eval(my_dict)
print(my_dict)
for k in my_dict:
  print(type(my_dict[k]))
  print(k,my_dict[k])

Then in the terminal/cmd line, you would write:

Running the code

python test.py --p '{"name": "Adam", "lr": 0.001, "betas": (0.9, 0.999)}'

Output

{'name': 'Adam', 'lr': 0.001, 'betas': (0.9, 0.999)}
<class 'str'>
name Adam
<class 'float'>
lr 0.001
<class 'tuple'>
betas (0.9, 0.999)

Answer 11

TLDR Solution: The simplest and quickest solution is as below:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-par", "--parameters",
                    default={},
                    type=str)
args = parser.parse_args()

In the parser.add_argument function:

1. Use a dictionary object for default object
2. str as the type

Then args.parameters will automatically be converted to a dictionary without any need for ast.literal.eval or json.loads.

Motivation: The methods posted by @Galuoises and @frankeye, appear to not work when the default is set as a json encoded dictionary such as below.

parser.add_argument("-par", "--parameters",
                required=False,  default="{\"k1\":v1, \"k2\":v2}",
                type=json.loads)

This is because

Answer 12

The following works just fine:

parser = argparse.ArgumentParser()
parser.add_argument("-par", "--parameters",
                required=False,  default={"k1a":"v1a","k2a":"v2a"},
                type=json.loads)
args = parser.parse_args()
print(str(parameters))

result:
{'k1a': 'v1a', 'k2a': 'v2a'}

For default value, the type should be dict since json.loads returns a dictionary, not a string, the default object should be given as a dictionary.

import argparse,json,sys
sys.argv.extend(['-par','{"k1b":"v1b","k2b":"v2b"}'])
parser = argparse.ArgumentParser()
parser.add_argument("-par", "--parameters",
                required=False,  default={"k1":"v1","k2":"v2"},
                type=json.loads)
args = parser.parse_args()
print(str(args.parameters))

result: 
{'k1b': 'v1b', 'k2b': 'v2b'}

Answer 13

 You could try:

$ ./script.py -i "{'name': 'img.png','voids': '#00ff00ff','0': '#ff00ff00','100%': '#f80654ff'}"

I haven't tested this, on my phone right now.

Edit: BTW I agree with @wim, I think having each kv of the dict as an argument would be nicer for the user.