Convert a string to list of list of dicts python

Question

I am new to python. I have a string which is taken from my sqlite database. I want to convert the string to a list of list of dicts. I tried ast and json library but it fails.

Here's the string:

a = '"[[{"plugin_type":"input","plugin":"mysql","dbname":"smackcoders","user":"root","password":"root","tbname":"agg_csv","host":"localhost","id":"mysql1","limit_count":5},{"plugin_type":"filter","plugin":"metrics","input_from":"mysql1","id":"metrics","field_name":"count_result", "field":"state","value":"kerala","action":"count","send_data_immediately":True},{"plugin_type":"output","plugin":"elastic","id":"elastic_search","input_from":"metrics","ind":"neww10","doc_typ":"sm23"}]]"'

Here the code i tried:

import ast
import json
a = '"[[{"plugin_type":"input","plugin":"mysql","dbname":"smackcoders","user":"root","password":"root","tbname":"agg_csv","host":"localhost","id":"mysql1","limit_count":5},{"plugin_type":"filter","plugin":"metrics","input_from":"mysql1","id":"metrics","field_name":"count_result", "field":"state","value":"kerala","action":"count","send_data_immediately":True},{"plugin_type":"output","plugin":"elastic","id":"elastic_search","input_from":"metrics","ind":"neww10","doc_typ":"sm23"}]]"'
a = a.replace("[[","[").replace("]]","]")
print(a)

# using json library- fails
# jdata = json.loads(a)
# for d in jdata:
#     for key, value in d.iteritems():
#         print (key, value)

#using ast library - it also fails
# res = [ast.literal_eval(x) for x in a]

I tried this link convert string to dict and convert str to list of list

But in my case, I have a list of list of dicts which is in the form of string. How to make it possible.

I want the same as the output but it should be in list of list of dicts.

Required Output:

[[{"plugin_type":"input","plugin":"mysql","dbname":"smackcoders","user":"root","password":"root","tbname":"agg_csv","host":"localhost","id":"mysql1","limit_count":5},{"plugin_type":"filter","plugin":"metrics","input_from":"mysql1","id":"metrics","field_name":"count_result", "field":"state","value":"kerala","action":"count","send_data_immediately":True},{"plugin_type":"output","plugin":"elastic","id":"elastic_search","input_from":"metrics","ind":"neww10","doc_typ":"sm23"}]]

If you do only general staff with sqlite, it would be better to use sqlite module, that wraps all low level operations. — Валерий Герасимов, May 30 '19 at 07:59
@Netwave, OPs problem wasn't using ast, it was that he has a string within a string which complicates matters. Your duplicate reference isn't quite right IMO — Will, May 30 '19 at 08:00

score 5 · Accepted Answer · answered May 30 '19 at 07:54

This should work.

import ast
a = '"[[{"plugin_type":"input","plugin":"mysql","dbname":"smackcoders","user":"root","password":"root","tbname":"agg_csv","host":"localhost","id":"mysql1","limit_count":5},{"plugin_type":"filter","plugin":"metrics","input_from":"mysql1","id":"metrics","field_name":"count_result", "field":"state","value":"kerala","action":"count","send_data_immediately":True},{"plugin_type":"output","plugin":"elastic","id":"elastic_search","input_from":"metrics","ind":"neww10","doc_typ":"sm23"}]]"'

print(ast.literal_eval(a.strip('"')))

Output:

[[{'dbname': 'smackcoders',
   'host': 'localhost',
   'id': 'mysql1',
   'limit_count': 5,
   'password': 'root',
   'plugin': 'mysql',
   'plugin_type': 'input',
   'tbname': 'agg_csv',
   'user': 'root'},
  {'action': 'count',
   'field': 'state',
   'field_name': 'count_result',
   'id': 'metrics',
   'input_from': 'mysql1',
   'plugin': 'metrics',
   'plugin_type': 'filter',
   'send_data_immediately': True,
   'value': 'kerala'},
  {'doc_typ': 'sm23',
   'id': 'elastic_search',
   'ind': 'neww10',
   'input_from': 'metrics',
   'plugin': 'elastic',
   'plugin_type': 'output'}]]

score 3 · Answer 2 · answered May 30 '19 at 07:55

You can use ast.literal_eval. You just have to make sure the string you feed it can be parsed. In this case it suffices to remove the initial and final quotes:

from ast import literal_eval
literal_eval(a[1:-1]) # or strip('"') as in rakesh' answer


[[{'dbname': 'smackcoders',
   'host': 'localhost',
   'id': 'mysql1',
   'limit_count': 5,
   'password': 'root',
   'plugin': 'mysql',
   'plugin_type': 'input',
   'tbname': 'agg_csv',
   'user': 'root'},
  {'action': 'count',
   'field': 'state',
   'field_name': 'count_result',
  ...

score 3 · Answer 3 · answered May 30 '19 at 07:56

Try this.

a = '"[[{"plugin_type":"input","plugin":"mysql","dbname":"smackcoders","user":"root","password":"root","tbname":"agg_csv","host":"localhost","id":"mysql1","limit_count":5},{"plugin_type":"filter","plugin":"metrics","input_from":"mysql1","id":"metrics","field_name":"count_result", "field":"state","value":"kerala","action":"count","send_data_immediately":True},{"plugin_type":"output","plugin":"elastic","id":"elastic_search","input_from":"metrics","ind":"neww10","doc_typ":"sm23"}]]"'

a=a.strip('"')
a=eval(a)

Convert a string to list of list of dicts python

3 Answers3