0

in my back-end code, I called R function from python with rpy2, and I have dictionary output from my R function, and I need to render it as JSON at my server endpoint. But output of json.dumps() for my dictionary is really not JSON object. I figured out because output of R function as StrVector, I have to stringify output data in dictionary then use json.dumps, but still couldn't get proper formatted JSON.

why json.dumps behaved stranged on data in the dictionary? what's the right way to get proper JSON structured data at the endpoint? any thought?

input data in dictionary:

here is the data in the dictionary that returned by R function. type of this dictionary is <class 'rpy2.robjects.vectors.StrVector'>

mydict = {"score":[72.188,62.0955,19.3374],"category":"medium",
"guidance":"text description","readiness":true,
"features_used":{"name":"gcstotal","value":null,
"range_value":[3.6667,5,6.3333,7.6667,9,10.3333,11.6667,13,14.3333],
"range_frequency":[0.0024,0,0.0049,0.0016,0.0073,0.0195,0.0098,0.0138,0.9406],"importance":0}}

update:

I just did quality check for mydict, it is <class 'rpy2.robjects.vectors.StrVector'>, so I tried below in my code:

json.dumps(str(mydict))

then I ended up below output. I called R function from python, and I used rpy2 to do so, but the output of json.dumps() for StrVector is not really json. why? any idea?

when I feed this dictionary to json.dumps(mydict ), I have this output:

"{\"score\":[72.188,62.0955,19.3374],\"category\":\"medium\",
\"guidance\":\"text description\",\"readiness_flag\":true,
\"features_used\":{\"name\":\"gcstotal\",\"value\":null,
\"range_value\":[3.6667,5,6.3333,7.6667,9,10.3333,11.6667,13,14.3333],
\"range_frequency\":[0.0024,0,0.0049,0.0016,0.0073,0.0195,0.0098,0.0138,0.9406],\"importance\":0}} \n"

I just don't understand why end up with non-JSON object from json.dumps. is it a problem from json.dumps or I used wrong way for jsonify dictionary at server endpoint? why I have above output? any idea? thanks

update 2::

after I tried @Victor S solution, it worked for pasted dictionary, but I am not sure I can decorate mydict object as follow as follow:

def post(self):
        if not request.get_json():
            return bad_request('No input data provided')
        raw_dict = request.json
        input_json = json.dumps(raw_dict)
        mydict = my_R_func(input_json)
        mydict = """mydict"""  ## inspired from @Victor S
        res = json.loads(str(mydict))
        res = json.dumps(res)
        return jsonify(res)

can I decorate mydict = my_R_func(input_json); mydict = """ mydict """ ? is there any way to do this?

jyson
  • 245
  • 1
  • 8
  • 27
  • Your process is not quite clear. Please provide a full code block with all `import` lines (not line snippets) of your Python script and JSON data. Code + data = [mcve]. Need to know the context more. Not quite clear why R is used at all. – Parfait May 08 '20 at 16:46
  • @Parfait what I did is call R function from python with `rpy2`, and I got dictionary like object but it is in `StrVector()` type. seems `pandas.rpy.common` might help, I got `ModuleNotFoundError: No module named 'pandas.rpy'` . any idea? I am just curious how below answer was generated. thanks a lot – jyson May 08 '20 at 16:48

2 Answers2

1

Try 

dict(zip(mydict.names, map(list, list(mydict))))

Credit to @CT Zhu for this answer

apparently this won't handle nested lists, so alternatively you can use rpy2 (pip install rpy2):

from rpy2.robjects import pandas2ri
pandas2ri.activate()

pandas2ri.ri2py(mydict)
Out[294]:
{"score":[72.188,62.0955,19.3374],"category":"medium",
"guidance":"text description","readiness":true,
"features_used":{"name":"gcstotal","value":null,
"range_value":[3.6667,5,6.3333,7.6667,9,10.3333,11.6667,13,14.3333],
"range_frequency":[0.0024,0,0.0049,0.0016,0.0073,0.0195,0.0098,0.0138,0.9406],"importance":0}}

rpy2 documentation

Rexovas
  • 469
  • 2
  • 9
  • how did you install pandas.rpy? I have this error: `ModuleNotFoundError: No module named 'pandas.rpy'`.I tried `dict(zip(mydict.names, map(list, list(mydict))))`, but it is not working. can you point me out how you make `pandas.rpy` worked? – jyson May 08 '20 at 16:31
  • I checked, and `pandas.rpy2` was deprecated. how did you get above your output? I am just surprised by how `pandas.rpy` worked for you. – jyson May 08 '20 at 16:35
  • I installed `pip install pandas==0.25.3`, still same error: `ModuleNotFoundError: No module named 'pandas.rpy'`. any idea how do you get above output? ` – jyson May 08 '20 at 16:46
  • I've updated my answer. Please see here https://pandas.pydata.org/pandas-docs/version/0.19/r_interface.html – Rexovas May 08 '20 at 16:53
  • please stay hold, let me try it before confirming your output is right. thanks – jyson May 08 '20 at 16:56
  • you have to `pip install rpy2` it is a different package. I was initially assuming the linked answer was still a valid solution. – Rexovas May 08 '20 at 16:57
  • yep, I installed `rpy2` from [here](https://www.lfd.uci.edu/~gohlke/pythonlibs/#pandas). your solution works partially, why there is error: `NameError: name 'true' is not defined`. seems logical value is not addressed. any possible update? – jyson May 08 '20 at 17:02
  • hmm... in Python, true is represented as True, and false is represented as False. It seems that because they are lowercase, and not quoted strings it is resulting in a NameError. Are you able to modify the R object before conversion? Alternatively you could define the variable `true = True` prior to conversion, but this would not be ideal. – Rexovas May 08 '20 at 17:07
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/213435/discussion-between-jyson-and-rexovas). – jyson May 08 '20 at 17:15
1

When you call json.dumps() you need to pass it a valid Python dict. For example your "readiness":true true here would throw a error as it's not a valid Python Boolean variable, but rather this is JSON syntax.

Try the following to get around this issue.

import json

mydict = """{"score":[72.188,62.0955,19.3374],"category":"medium",
"guidance":"text description","readiness":true,
"features_used":{"name":"gcstotal","value":null,
"range_value":[3.6667,5,6.3333,7.6667,9,10.3333,11.6667,13,14.3333],
"range_frequency":[0.0024,0,0.0049,0.0016,0.0073,0.0195,0.0098,0.0138,0.9406],"importance":0}}"""

# convert to string
mydict_str = json.loads(mydict)

# to json obj.
tojson = json.dumps(mydict_str , indent=4)
print(tojson)
  • do you think is there any way we can convert `StrVector` to normal dictionary? why we envelope `mydict inside """"""`? – jyson May 08 '20 at 17:35
  • can I do res = """mydict"""? because in mydict was output from my function, seems I have to decorate whatever I got from my function inside """ """. how can I decorate output object inside """ """? – jyson May 08 '20 at 17:42
  • I got above `mydict` after I printed out how `mydict` looks like, how am I gonna decorate output object, like `mydict`? your solution worked but I have to wrap up non-printed `mydict` inside """ """. any idea? – jyson May 08 '20 at 17:55
  • please take a look my `update 2` on my post. Is that doable? thanks – jyson May 08 '20 at 18:04
  • @jyson what does "raw_dict" print out? –  May 08 '20 at 22:04
  • If your "raw_dict" or "input_json" appears as in your first example "input data in dictionary:", you should be able to call json.load(raw_dict) followed by json.dumps(mydict ) to serialize to JSON. –  May 08 '20 at 23:14