How to write a DRY wrapper that works with intellisense in python

Question

I'm writing a python tool with modules at different 'levels':

• A low-level module, that can do everything, with a bit of work
• A higher level module, with added "sugar" and helper functions

I would like to be able to share function signatures from the low-level module to the higher one, so that intellisense works with both modules.

In the following examples, I'm using the width and height parameters as placeholders for a pretty long list of arguments (around 30).

I could do everything explicitly. This works, the interface is what I want, and intellisense works; but it's very tedious, error prone and a nightmare to maintain:

# high level function, wraps/uses the low level one
def create_rectangles(count, width=10, height=10):
  return [create_rectangle(width=width, height=height) for _ in range(count)]

# low level function
def create_rectangle(width=10, height=10):
  print(f"the rectangle is {width} wide and {height} high")

create_rectangles(3, width=10, height=5)

I could create a class to hold the lower function's parameters. It's very readable, intellisense works, but the interface in clunky:

class RectOptions:
  def __init__(self, width=10, height=10) -> None:
    self.width = width
    self.height = height

def create_rectangles(count, rectangle_options:RectOptions):
  return [create_rectangle(rectangle_options) for _ in range(count)]

def create_rectangle(options:RectOptions):
  print(f"the rectangle is {options.width} wide and {options.height} high")

# needing to create an instance for a function call feels clunky...
create_rectangles(3, RectOptions(width=10, height=3))

I could simply use **kwargs. It's concise and allows a good interface, but it breaks intellisense and is not very readable:

def create_rectangles(count, **kwargs):
  return [create_rectangle(**kwargs) for _ in range(count)]

def create_rectangle(width, height):
  print(f"the rectangle is {width} wide and {height} high")

create_rectangles(3, width=10, height=3)

What I would like is something that has the advantages of kwargs but with better readability/typing/intellisense support:

# pseudo-python

class RectOptions:
  def __init__(self, width=10, height=10) -> None:
    self.width = width
    self.height = height

# The '**' operator would add properties from rectangle_options to the function signature
# We could even 'inherit' parameters from multiple sources, and error in case of conflict
def create_rectangles(count, **rectangle_options:RectOptions):
  return [create_rectangle(rectangle_options) for idx in range(count)]

def create_rectangle(options:RectOptions):
  print(f"the rectangle is {options.width} wide and {options.height} high")

create_rectangles(3, width=10, height=3)

I could use code generation, but I'm not very familiar with that, and it seems like it would add a lot of complexity.

While looking for a solution, I stumbled upon this reddit post. From what I understand, what I'm looking for is not currently possible, but I really hope I'm wrong about that

I've tried the the docstring_expander pip package, since it looks like it's meant to solve this problem, but it didn't do anything for me (I might be using it wrong...)

I don't think this matters but just in case: I'm using vscode 1.59 and python 3.9.9

Answer 1

It is indeed not currently possible. Intellisense is something that IDE's do to make your life easier but its limited since it cant run your code... so dynamic named parameters wont work. Its either you are explicit on what params you accept def test(param1: str, param2: str) or you go dynamic completely def test(**kwargs). You cant have both dynamic and explicit.

Just to add to the convo another option is to use TypedDict:

from typing import TypedDict

class RectOptions(TypedDict):
    width: int
    height: int

def create_rectangles(count, opts: RectOptions):
  print(count)
  print(opts)

create_rectangles(0, {'width': 1, 'height': 2})

simplified since we only really care about the function def.

This way its a dict but typed so if you try pass something that is not in the type IDE's will warn. (Pycharm at least does)

Answer 2

You have two possible ways to achieve that, which would let you have optional arguments and inherit multiple arguments "sources", as you call them.

• Note that this has been asked multiple times (e.g. 1, 2, 3, 4). In spite of what's coming up from past answers, I think it's actually fair to say it's somewhat possible.
  I'm going to offer a suggestion I couldn't find in those past answers, as well as accomodating one past answer to your case.

1. Using @dataclass + kw_only attribute (available in Python 3.10)

dataclasses 3.10 added the kw_only attribute. To let @dataclass achieve what you'd like, you'll have to use this new attribute, otherwise you'll either just get inheritance errors, or an incomplete Intellisense support (I'll expand below, but you can read more in this answer).

It works as follows:

Types:

from dataclasses import dataclass

@dataclass
class RequiredProps:
    # all of these must be present
    width: int
    height: int

@dataclass(kw_only=True) # <-- the kw_only attribute is available in python 3.10
class OptionalProps:
    # these can be included or they can be omitted
    colorBackground: str = None
    colorBorder: str = None

@dataclass
class ReqAndOptional(RequiredProps, OptionalProps):
    pass

Functions:

Option #1

def create_rectangles(count, args: ReqAndOptional):
   return [create_rectangle(args.width, args.height) for _ in range(count)]

def create_rectangle(width, height):
   print(f"the rectangle is {width} wide and {height} high")

create_rectangles(1, ReqAndOptional(width=1, height=2))

Option #2

def create_rectangles(count, args: ReqAndOptional):
   return [create_rectangle(args) for _ in range(count)]

def create_rectangle(args: RequiredProps):
   print(f"the rectangle is {args.width} wide and {args.height} high")

create_rectangles(1, ReqAndOptional(width=1, height=2))

Note that once you type ReqAndOptional, Intellisense will autocomplete the dict properties:

2. Using TypedDict

You can accomodate this past answer to your case:

Types:

import typing

class RequiredProps(typing.TypedDict):
    # all of these must be present
    width: int
    height: int

class OptionalProps(typing.TypedDict, total=False):
    # these can be included or they can be omitted
    colorBackground: str
    colorBorder: str

class ReqAndOptional(RequiredProps, OptionalProps):
    pass

Functions:

Option #1

def create_rectangles(count, args: ReqAndOptional):
   return [create_rectangle(args['width'],args['height']) for _ in range(count)]

def create_rectangle(width, height):
   print(f"the rectangle is {width} wide and {height} high")

create_rectangles(1, {'width':1, 'height':2, 'colorBorder': '#FFF'})

Option #2

def create_rectangles(count, args: ReqAndOptional):
   return [create_rectangle(args) for _ in range(count)]

def create_rectangle(args: RequiredProps):
   print(f"the rectangle is {args['width']} wide and {args['height']} high")

create_rectangles(1, {'width':1, 'height':2, 'colorBorder': '#FFF'})

Note that you get autocompletion on both places:

On the calling side, Intellisense will show you the two optional arguments:

And on the called side, once you type args[], Intellisense will autocomplete the dict properties:

Closing Notes

• What happens if you don't use kw_only supported in Python 3.10? as mentioned above, you'll either face inheritance problems or incomplete Intellisense autocompletion. Let's see each of them:

  1. First, try to omit the kw_only attribute, and run the program. You'll get the error:

  non-default argument 'width' follows default argument

  That's because attributes are combined by starting from the bottom of the MRO. You can refer to this answer to better understand what happens.

  2. Second, you might want to solve the error we got in section 1 by trying to use field(default=None, init=False) (as suggested in this other answer), but this would result in incomplete Intellisense autocompletion, as in the image below.

• What happens if you'd like to replace typing.TypedDict with typing.NamedTuple? again, you'll get an incomplete Intellisense autocompletion (see here why):