3

I'm using python 3 for an application. For that utility, I need to pass command line arguments as following, 

python3 -m com.xxx.executor -Denvironment=dev -Dtoggle=False

Both the parameter environment and toggle are present in a property file too. If the value is specified in command line, it should override what is present on property file.

I'm basically a java guy and in java, the properties passed in the form -Dkey=value will be set as system property. Then these properties can be read from code as System.getProperty(key, defaultVal).

But when I try the same in python 3, it didn't work.

After referring python docs, it seems to me like the sys._xoptions are suitable for my requirement. 

 python3 -Xenvironment=dev -Xtoggle=False -m com.xxx.executor

Then read the properties using, sys._xoptions

I'm using Cpython. The aim of the thread is to ensure that, the way I'm proceeding is right or not. Or is there any other better ways in python to implement the same.

Python veterans, please guide !

appu
  • 538
  • 10
  • 24
  • from `import this`, _flat is better than nested_. While Python encourages namespacing, you probably don't want `com.xxx.executor`. Java encourages nesting, but you really don't need that in Python. – Wayne Werner Jul 23 '16 at 12:05
  • The problem is that, I'm invoking the framework entry class(`com.xxx.executor`) from Jenkins build, where the module is pre installed in agent machine using a wheel package in its `{python.sys.path}\Lib\site-pacakges\com\xxx\executor`. So how can I use `import this` then ? – appu Jul 23 '16 at 17:22
  • Well, if someone *else* created a terribly nested package, then you're obligated to use it, but if you have any control over the nesting of the package you should fix that. – Wayne Werner Jul 24 '16 at 12:29
  • Okay, I suggested to remove the nested packaging to the framework owners. – appu Aug 05 '16 at 05:06

2 Answers2

0

For argument parsing, I use the argparse module (docs) to define which are valid named and/or positional arguments.

There are third-party modules as well such as click and docopt. You should use what you feel most comfortable with and whether or not you can use third-party modules. The click documentation contains a (possibly biased) comparison between it, argparse and docopt.

I've never used sys._xoptions, nor did I know of its existence. Seems a bit strange that a function starting with an underscore (normally used to indicate a "private" function) is mentioned in the docs. Perhaps someone else can shed some light on this.

For the parsing of a property file, I use the configparser module (docs). Of course, you could opt for a JSON or YAML config file if you'd prefer that.

That said, you'll have to come up with the necessary code to overrule properties when specified as arguments (though that shouldn't be too difficult).

Community
  • 1
  • 1
DocZerø
  • 8,037
  • 11
  • 38
  • 66
  • OK. For property file parsing, I'm currently using `configparser`. Problem is with how to pass a property as generic `key=value` from command line. I think, I can use `argparse`. But I do have arount 25+ properties of which 2 or 3 are used in one execution. So thought of a generic way to pass `key=value` from command line(`-Xkey=value), so that I don't need to write 25+ different `parser.add_argument(key)`. I'll work out on `argparse` and let u know. Also, third party libraries and not allowed to use. – appu Jul 23 '16 at 17:33
  • You don't *have* to use `argparse` to parse the commandline options, you could very well reference `sys.argv`. `argparse` allows you to define which arguments you support, configure valid values for them etc. – DocZerø Jul 23 '16 at 17:35
  • If all your arguments are more or less the same (i.e. only differ in name, but same datatype etc), you could just as easily wrap the `add_argument()` call in a `for` loop with a static list of values, e.g. `for arg in ['key1', 'key2',…]` – DocZerø Jul 23 '16 at 17:38
  • Datatypes of the values may differ. But those will be simple objects of types `str`, `int`, `bool`, `float` etc and not complex data structures like dictionary, lists, tuples etc. – appu Jul 23 '16 at 17:42
  • I'll try the mentioned `argparse` option and get back to you – appu Jul 23 '16 at 17:44
  • I tried with `argparse` and it worked like a charm ! – appu Aug 05 '16 at 05:06
0

From the docs on -X args

Reserved for various implementation-specific options. CPython currently defines the following possible values:

That means you probably shouldn't be hijacking these for your own purposes. As Kristof mentioned, argparse is a pretty reasonable choice. Since you want both a file and command line arguments, here's a quick example using a json file-based config:

import json                                                 
import argparse                                             

argparser = argparse.ArgumentParser()                       
argparser.add_argument('--environment')                     
argparser.add_argument('--toggle', action='store_true')     
try:                                                        
    with open('config.json') as f:                          
        args = json.load(f)                                 
except (IOError, ValueError) as e:                          
    # If the file doesn't exist or has invalid JSON         
    args = {}                                               
args.update(vars(argparser.parse_args()))                   
print(args)

There are other possible alternatives for the file-based config, like the configparser module.

Wayne Werner
  • 49,299
  • 29
  • 200
  • 290
  • Please see https://docs.python.org/3.4/library/sys.html#sys._xoptions from python docs. So is it not safe to use with Cpython? Please share your suggestions – appu Jul 23 '16 at 17:46
  • @appu if you carefully read that documentation, it says _This is a CPython-specific way of accessing options passed through -X. Other implementations may export them through other means, or not at all._ and if you click on the `-X` it will send you to the docs that I linked to. You *could* use it, but you're asking for trouble. Better to use an official/supported method for getting parameters from the command line/files. – Wayne Werner Jul 24 '16 at 12:20