Python replace values in unknown structure JSON file

Question

Say that I have a JSON file whose structure is either unknown or may change overtime - I want to replace all values of "REPLACE_ME" with a string of my choice in Python.

Everything I have found assumes I know the structure. For example, I can read the JSON in with json.load and walk through the dictionary to do replacements then write it back. This assumes I know Key names, structure, etc.

How can I replace ALL of a given string value in a JSON file with something else?

something like this? https://stackoverflow.com/questions/14882138/replace-value-in-json-file-for-key-which-can-be-nested-by-n-levels — Jean-François Fabre, May 31 '18 at 19:24

Tony Suffolk 66 · Answer 1 · 2018-06-01T22:45:45.043

This function recursively replaces all strings which equal the value original with the value new.

This function works on the python structure - but of course you can use it on a json file - by using json.load

It doesn't replace keys in the dictionary - just the values.

def nested_replace( structure, original, new ):
    if type(structure) == list:
        return [nested_replace( item, original, new) for item in structure]

    if type(structure) == dict:
        return {key : nested_replace(value, original, new)
                     for key, value in structure.items() }

    if structure == original:
        return new
    else:
        return structure

d = [ 'replace', {'key1': 'replace', 'key2': ['replace', 'don\'t replace'] } ]
new_d = nested_replace(d, 'replace', 'now replaced')
print(new_d)
['now replaced', {'key1': 'now replaced', 'key2': ['now replaced', "don't replace"]}]

score 3 · Answer 2 · answered May 31 '18 at 19:28

I think there's no big risk if you want to replace any key or value enclosed with quotes (since quotes are escaped in json unless they are part of a string delimiter).

I would dump the structure, perform a str.replace (with double quotes), and parse again:

import json

d = { 'foo': {'bar' : 'hello'}}

d = json.loads(json.dumps(d).replace('"hello"','"hi"'))

print(d)

result:

{'foo': {'bar': 'hi'}}

I wouldn't risk to replace parts of strings or strings without quotes, because it could change other parts of the file. I can't think of an example where replacing a string without double quotes can change something else.

There are "clean" solutions like adapting from Replace value in JSON file for key which can be nested by n levels but is it worth the effort? Depends on your requirements.

Nicolò Gasparini · Answer 3 · 2018-06-01T07:10:32.623

2

Why not modify the file directly instead of treating it as a JSON?

with open('filepath') as f:
    lines = f.readlines()
for line in lines:
    line = line.replace('REPLACE_ME', 'whatever')
    with open('filepath_new', 'a') as f:
        f.write(line)

edited Jun 01 '18 at 07:10

answered May 31 '18 at 19:28

Nicolò Gasparini

2,228
2
24
53

score 2 · Answer 4 · edited Dec 02 '20 at 19:55

2

Well that depends, if you want to place all the strings entitled "REPLACE_ME" with the same string you can use this. The for loop loops through all the keys in the dictionary and then you can use the keys to select each value in the dictionary. If it is equal to your replacement string it will replace it with the string you want.

search_string = "REPLACE_ME"
replacement = "SOME STRING"
test = {"test1":"REPLACE_ME", "test2":"REPLACE_ME", "test3":"REPLACE_ME", "test4":"REPLACE_ME","test5":{"test6":"REPLACE_ME"}}


def replace_nested(test):

    for key,value in test.items():

        if type(value) is dict:
            replace_nested(value)
        else:
            if value==search_string:
                test[key] = replacement


replace_nested(test)
print(test)

edited Dec 02 '20 at 19:55

Jean-François Fabre

137,073
23
153
219

answered May 31 '18 at 19:33

MicahB

134
7

This assumes the JSON has no nesting and `test` is a dictionary, not a JSON file – s g May 31 '18 at 19:59
I added a recursive function to include nested dicts. Can you convert the JSON to a dictionary? – MicahB May 31 '18 at 20:10
it doesn't need to be a dictionary - even a list can be iterated through. – Tony Suffolk 66 Jun 01 '18 at 19:46

s g · Accepted Answer · 2018-06-01T20:07:26.283

You could load the JSON file into a dictionary and recurse through that to find the proper values but that's unnecessary muscle flexing.

The best way is to simply treat the file as a string and do the replacements that way.

json_file = 'my_file.json'
with open(json_file) as f:
    file_data = f.read()

file_data = file_data.replace('REPLACE_ME', 'new string')
<...>

with open(json_file, 'w') as f:
    f.write(file_data)

json_data = json.loads(file_data)

From here the file can be re-written and you can continue to use json_data as a dict.

score 0 · Answer 6 · edited Jun 20 '20 at 09:12

To solve this problem in a dynamic way, I have obtained to use the same json file to declare the variables that we want to replace.

Json File :

{
"properties": {
"property_1": "value1",
"property_2": "value2"
},
"json_file_content": {
   "key_to_find": "{{property_1}} is my value"
   "dict1":{
       "key_to_find": "{{property_2}} is my other value"
   }
}

Python code (references Replace value in JSON file for key which can be nested by n levels):

import json    
def fixup(self, a_dict:dict, k:str, subst_dict:dict) -> dict:
"""
function inspired by another answers linked below
""" 
    for key in a_dict.keys():
        if key == k:
            for s_k, s_v in subst_dict.items():
                a_dict[key] = a_dict[key].replace("{{"+s_k+"}}",s_v)
        elif type(a_dict[key]) is dict:
            fixup(a_dict[key], k, subst_dict)
# ...
file_path = "my/file/path"
if path.exists(file_path):
   with open(file_path, 'rt') as f:
   json_dict = json.load(f)
   fixup(json_dict ["json_file_content"],"key_to_find",json_dict ["properties"])
   print(json_dict) # json with variables resolved
else:
   print("file not found")

Hope it helps

Python replace values in unknown structure JSON file

6 Answers6

Linked