0

Hello and good year 2017 to everyone!

I have issues to send a quiet complex json to a cloudant DB. As far as I understand (and it's not really far because i'm a total newbie), the only way to send a file on the cloudant DB with cloudant for python is to make a variable "data"

data = {
    '_ID': 'userInfo',
    'Name': 'Manu'
    'Age': 23,
    'Hobbies': ['cloudant', 'python', 'dockers']
    }

and to use the command

my_document = my_database.create_document(data)

which succesfully create a file in the DB.

BUT it looks like everything goes trickier when the data is not inside a dict, because at somepoint the .create_document() method try to extract keys and values and it goes wild when you give him a list link to console error: https://www.jepix.fr/images/cloudanterror.png

but it's not a regular list that I try to give him, it's a magnificent list of dicts :-) 

[
   {dict1},
   {dict2},
   {dict3},
   {dict4},
   { },
   { },
   { },
   { },
   { }
]

with marvelous dicts inside the dicts :-D I think I need a way to convert the list of dicts into a dict of dicts and I ABSOLUTELY HAVE TO keep this dict structure because it is use by other colleagues' templates, parser etc. or my apprentice's bottom will be kicked ruthlessly! 

import cloudant
import json
from cloudant.client import Cloudant


client = Cloudant("USR", "PWD", url)
client.connect()

# Client tasks ##########################################
session = client.session()
print 'Username: {0}'.format(session['userCtx']['name'])
print 'Databases: {0}'.format(client.all_dbs())

my_db = client['scenario-json']

with open('scenarioGenerated.json', 'r') as fp:
    json_str = fp.read()

json_dict = json.loads(json_str)
my_doc = my_db.create_document(json_dict)



client.disconnect()

I have tried total newbie's desperate move like casting the list into dict with dict() function but I got an argument error telling me I have not enough argument to call this function (at this point I awfully accepted it and searched for another way out)

I also tried the json library for python with json.load/.loads/.dump/.dumps but it's the same problem as it convert the json to a string (looping back on the "str" object has no "get" attribute error)

I saw maybe a solution on an overflow post: Python dump dict to json file

But I need to do the opposite, like reading the json and fill in a dict? I don't think it's the cleverest moves as the dict-filling process can loose the dict hierarchy...

So imo, if you have a nice way to convert the list of dicts into a dict of dicts it would be so nice!

Sorry for the long post and very noobish question but i'm like a plankton in a sea of whales and it's not a good situation :-p

Thank you very much.

Community
  • 1
  • 1
Manu
  • 352
  • 1
  • 4
  • 14
  • 1
    Please show the code you have tried, so that u can get help here – Siva Cn Jan 03 '17 at 15:37
  • Don't know anything about Cloudant, but the [documentation](https://docs.cloudant.com/document.html) says that other than the mandatory `_id` and `_rev` field, documents can contain just about any other JSON you want. So wouldn't something like `{ "_id": "foo", "_rev": "bar", "my_array": [ ... ] }` be better? – ThisSuitIsBlackNot Jan 03 '17 at 15:46
  • @SivaCn done, this is supposed to work because that's what I did when I was training with socket, I used to loads and dumps json as a string to send it through sockets. – Manu Jan 03 '17 at 15:54
  • @ThisSuitIsBlackNot you mean building a dict with: data = { "_id": "killme", "my_array" : the_json_list_of_dicts } – Manu Jan 03 '17 at 15:56
  • @Manu Yep, exactly. That way you don't have to change your data structure, you just nest it inside a dictionary. – ThisSuitIsBlackNot Jan 03 '17 at 15:58

2 Answers2

1

Easy but quite unnecessary would be to just put a dict around the whole list. So a dict with one key and one element. That one element being your list... I Dont see why you have this structure though and why this is how it needs to be, so its hard for me to help out any other way.

Schorsch
  • 171
  • 9
  • I have to keep this structure because it's a simulation scenario where each elements of the json goes through a template, then the template's var are used by the program to create a payload sent to a mqtt broker :/ I'm a begginer and it sounds so crazy to be on this but thanks for your comment I'm going to try that and keep you in touch :) – Manu Jan 03 '17 at 15:37
0

Do you need to create all the data in a single call to the create_document() function, or can you make multiple calls? If you can do the second, I would write something like this:

documents = [ {dict1}, {dict2}, {dict3}, ...]
for document in documents:
    my_document = my_database.create_document(document)
    # Do some more processing.

This creates one document for each document in the list.

Following up on the comment by thissuitisblacknot, you might be able to do something like this:

documents = [ {dict1}, {dict2}, {dict3}, ...]
data = { "docs": documents }
my_database.bulk_create_document(data) # Guessed at bulk_create_document()
Simon Callan
  • 3,020
  • 1
  • 23
  • 34
  • 1
    There's also apparently a way to do [bulk operations](https://docs.cloudant.com/document.html#bulk-operations) to create multiple documents at once. – ThisSuitIsBlackNot Jan 03 '17 at 15:42
  • Yes I'm supposed to because each data inside each dicts are specific to a device and I can't separate devices :/ – Manu Jan 03 '17 at 15:47
  • Ok thank you so much guys Simon and @ThisSuitIsBlackNot it looks like it work and moreover, once the file is upload I can edit it haha :-) Have a good evening! – Manu Jan 03 '17 at 16:04
  • And it works with regular command without bulk (and it's nice because I don't know bulk yet) And it's even easier to load back the json as it's just a value of a key, wonderful ^_^ Thank you again! – Manu Jan 03 '17 at 16:13