0

I am using python 3.10.4. I have a string which is as follows:

str_obj = "[(190229461780583, 'main'), (302030571463836, 'feature/BRANCH_livestream'), (1071064128966159, 'feature/BRANCH_sensitive'), (1137211553786277, 'main'), (1283366887974580, 'feature/add_sql_vm'), (1492751472739439, 'feature/BRANCH-2977'), (2662272892040840, 'main'), (4078787696930326, 'main')]"

I need to convert it to a json object. I need it in json format for further data extraction. Following this link I used json.loads() to convert it from string to json:

json_obj = json.loads(str_obj)

However I am getting the error:

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I checked this link but it is not really related to my problem.

I cannot simply drop "" from str_obj as this is the value I am receiving from an environment variable sent to me from a DevOps pipeline.

I would like to know what the root cause problem is and how I can resolve it.

E. Erfan
  • 1,239
  • 19
  • 37
  • 1
    The error says `line 1 column 2`, not `line 1 column 1`. It happens because the string is **not valid JSON**. Please make sure you [know the JSON specification](https://www.json.org/json-en.html). It is **not** the same as Python syntax. "I cannot simply drop "" from str_obj" In the code that you show, `str_obj` **does not** have quotation marks surrounding it - that's part of the Python syntax to indicate that you have a string, not part of the string contents. – Karl Knechtel May 27 '22 at 21:08
  • 1
    "this is the value I am receiving from an environment variable sent to me from a DevOps pipeline." Please talk to whoever is responsible for that end of the pipeline, and make sure you both have the same understanding of what is supposed to be sent and how it should be interpreted. It's possible that this should be considered a bug on their end. – Karl Knechtel May 27 '22 at 21:09
  • The other end is also me, I am getting a response by request.get() in python. The API response is too long and I only need these two features. So I made two arrays called ids and branches, and make the structure above by list(zip(ids, branches)). I later sent this value in a job in Azure DevOps. Now I am capturing it in another job and need to extract only ids, whose branch has the value main. – E. Erfan May 27 '22 at 21:19
  • 1
    "So I made two arrays called ids and branches, and make the structure above by list(zip(ids, branches))." Then it is probably better to make JSON out of this structure explicitly (using `json.dumps`, or `json.dump` if you have a stream to feed it to directly). If you are receiving this data as described, that implies that it was converted to string via `str`. While it's possible to work with such data, it's not the Web standard and it isn't JSON. – Karl Knechtel May 27 '22 at 21:24
  • @KarlKnechtel One side question, is it possible to filter the values in this string without converting it to json? I need to get all ids, whose branch value (2nd value in each pair) equals 'main'. – E. Erfan May 27 '22 at 21:24
  • 1
    Why not implement that on the other side, where you already have the data structure? That said, you probably could do it, but deserializing the data first will definitely be less painful. – Karl Knechtel May 27 '22 at 21:26
  • @KarlKnechtel just to double check, you are refering to json.dumps, not dump, right? After applying json.dumps at the other end to the list, I get this format: '[[190229461780583, "main"], [302030571463836, "feature/e2e_livestream"], [1071064128966159, "feature/schema_mdd_sensitive"], [1137211553786277, "main"], [1283366887974580, "feature/add_sql_vm"], [1492751472739439, "feature/ORGTHDATAMA-2977"], [2662272892040840, "main"], [4078787696930326, "main"]]' It seems that this is still not valid json. Isn't it? – E. Erfan May 27 '22 at 21:34
  • 1
    "you are refering to json.dumps, not dump, right?" `.dumps` creates a string; `.dump` writes directly to a stream - which some web APIs might offer you. "It seems that this is still not valid json. Isn't it?" It is not the *usual way of* using JSON, but it is entirely valid JSON. Again, please read the spec. – Karl Knechtel May 27 '22 at 21:36

3 Answers3

1

Probably something like this would help you if you wanted to make json out of this string. Surely the best option would be to fix the input data

str_obj = "[(190229461780583, 'main'), (302030571463836, 'feature/BRANCH_livestream'), (1071064128966159, 'feature/BRANCH_sensitive'), (1137211553786277, 'main'), (1283366887974580, 'feature/add_sql_vm'), (1492751472739439, 'feature/BRANCH-2977'), (2662272892040840, 'main'), (4078787696930326, 'main')]"
# clear the input string
str_obj = str_obj.replace('[','').replace(']','').replace('(','').replace(')','').replace('\'','').replace(' ','').split(',')
# create output string
output = {}
# Create pairs of key:value
str_to_list = [str_obj[i:i+2] for i in range(0, len(str_obj), 2)]
# append to output dict
for e in str_to_list:
    output[e[0]] = e[1]

# You can easly create json form dict
json_object = json.dumps(output)
wordinone
  • 131
  • 1
  • 9
  • your code is working. Can you explain a little this part: str_obj[i:i+2]? What is the logic of +2? – E. Erfan May 27 '22 at 22:15
  • 1
    Sure, this is how I only want to select one pair to save it as a list which I can then convert to a key:value pair in dict. Try playing with the values to better understand the algorithm. – wordinone May 27 '22 at 22:35
0

That isn't json, json uses key value pairs. Did you want ast.literal_eval?

import ast
ast.literal_eval(str_obj)
# [(190229461780583, 'main'), (302030571463836, 'feature/BRANCH_livestream'), (1071064128966159, 'feature/BRANCH_sensitive'), (1137211553786277, 'main'), (1283366887974580, 'feature/add_sql_vm'), (1492751472739439, 'feature/BRANCH-2977'), (2662272892040840, 'main'), (4078787696930326, 'main')]
Kraigolas
  • 5,121
  • 3
  • 12
  • 37
  • is `ast.lit_eval` faster than `eval`? – rv.kvetch May 27 '22 at 21:06
  • 2
    @rv.kvetch It is much more restricted, which is the reason to recommend it. `eval` is a **major security risk** and essentially never belongs in a public-facing answer on Stack Overflow (except of course to explain the security risks); the very few people who have a valid use case for it will generally be experts capable of solving the problem themselves. See also e.g. https://stackoverflow.com/questions/15197673/using-pythons-eval-vs-ast-literal-eval. – Karl Knechtel May 27 '22 at 21:11
  • "json uses key value pairs." Valid JSON does not have to do so; a nested list is perfectly valid, for example. However, JSON syntax uses only `[]` and not `()` for sequences (what the JSON spec calls "arrays"), and it only uses `""` and not `''` to enclose strings. – Karl Knechtel May 27 '22 at 21:14
  • I tried ast.literal_eval() too, but I get the error: TypeError: unhashable type: 'list'. – E. Erfan May 27 '22 at 21:16
  • The other end is also me, I am getting a response by request.get() in python. The API response is too long and I only need these two features. So I made two arrays called ids and branches, and make the structure above by list(zip(ids, branches)). I later sent this value in a job in Azure DevOps. Now I am capturing it in another job and need to extract only ids, whose branch has the value main. – E. Erfan May 27 '22 at 21:20
0

The value of your str_obj is simply not a valid json format at all. You can use a JSON parser online to verify the syntax you need to turn into a valid JSON : Online JSON parser

Chuk Ultima
  • 987
  • 1
  • 11
  • 21