0

I am using JSON to send data from Python to R (note: I'm much more familiar with R than Python). For primitives, the json module works great. For many other Python objects (e.g. numpy arrays) you have to define a custom encoder, like in this stack overflow answer. However, that requires you to pass the encoder as an argument to json.dumps, which doesn't work that well for my case. I know there are other packages like json_tricks that have much more advanced capabilities for JSON serialization, but since I don't have control over what Python distribution a user has I don't want to rely on any non-default modules for serializing objects to JSON.

I'm wondering if there is a way to use contextlib decorators to define additional ways for serializing JSON objects. Ideally, I'm looking for a way that would allow users to overload some standard function standard_wrapper that I provide to add new methods for their own classes (or types from modules that they load) without requiring them to modify standard_wrapper. Some psuedocode below:

import json

def standard_wrapper(o):
  return o


obj = [44,64,13,4,79,2,454,89,0]

json.dumps(obj)
json.dumps(standard_wrapper(obj))

import numpy as np
objnp = np.sort(obj)

json.dumps(objnp)  # FAILS

@some_decorator_to_overload_standard_wrapper
# some code

json.dumps(standard_wrapper(objnp))  # HOPEFULLY WORKS

This is essentially function overloading by type---I've seen examples for overloading by arguments in Python, but I don't see how to do it by type.

EDIT I was mixing up decorators with contextlib (which I had only ever seen used a decorator).

mikeck
  • 3,534
  • 1
  • 26
  • 39
  • It's a mystery to me why you were wondering about using `contextlib` for this—what did you have in mind? Regardless, the answers to the question [**_Making object JSON serializable with regular encoder_**](https://stackoverflow.com/questions/18478287/making-object-json-serializable-with-regular-encoder) might be helpful. – martineau Jul 05 '17 at 03:00
  • I don't want to create an encoder for a custom class, I want a framework that would allow users to define their own methods for serializing an object that won't conflict with existing methods and don't require specifying custom `cls` arguments to `json.dumps`. I guess I'm looking for a python equivalent to S3 methods in R. – mikeck Jul 05 '17 at 03:32
  • It doesn't sound to me like your read my answer to the linked question. It shows how to hook into the json encoding scheme and do anything you want. The code uses `pickle` as an example, because it can serialize many common Python types without doing anything further. It's an extensible system, so many authors add methods to support it to their classes. I think it would fairly straightforward to make your own "framework" that used user supplied "methods" to do things, but doing so is going to require some way to associate them with the proper objects—using `cls` is an obvious choice to do this – martineau Jul 05 '17 at 04:19
  • @martineau, I did read your link but it requires that the default encoder is replaced. This would only work if I knew in advance every possible data type that I need to support. I am looking for a solution that will allow users to *extend* the encoder (either the default encoder or one that I provide) to support additional data types without needing to completely overwrite the encoder. – mikeck Jul 05 '17 at 17:16
  • Sorry, I still don't think you're getting it—as it doesn't necessarily require you to need to know every possible data type in advance. The patch is only invoked when the stock encoder encounters something (of a type) that it doesn't already know how to handle. This basically gives you the "hook" needed to do whatever you want when it happens, which could include looking at what your users have provided to see if there's something that could to handle the type of object in question. I would show you how to do this, but I don't have `numpy` installed (nor wish to do so). – martineau Jul 05 '17 at 18:21
  • 1
    As far as overloading functions by type goes, what don't you understand about doing it? Instead of checking the value of the argument passed, just check its type (with `isinstance()`). – martineau Jul 05 '17 at 18:25

1 Answers1

1

It's easy to use singledispatch from functools module to overload a function by type, as shown in this answer to a different post. However, a simpler solution that may fit my needs is to create a dictionary of functions where the keys correspond to the object type. 

import numpy
func_dict = {}
a = [2,5,2,9,75,8,36,2,8]
an = numpy.sort(a)

func_dict[type(an)] = lambda x: x.tolist()
func_dict[type(a)] = lambda x: x

import json
json.dumps(func_dict[type(a)](a))
json.dumps(func_dict[type(an)](an))

Adding support for another type is achieved by adding another function to the dictionary.

mikeck
  • 3,534
  • 1
  • 26
  • 39
  • 1
    This is simpler? The solution for your users is that they don't need to know what the type they've created is, but they do need to know the appropriate type to convert to in order to serialize. It seems to me that knowing the appropriate serializable type is harder to discover than the type of the datastructure you've created. The simpler answer is the first option illustrated in this article: https://hynek.me/articles/serialization/ The singledispatch method, also in that article, answers your original question, using decorators to "overload a function by type" AKA dynamic dispatch. – Davos Nov 23 '17 at 12:52