2

I have 25 json files in a folder, named 0.json through 24.json, and I am trying to batch open and rename a perimeter "image" inside of each, which currently all have a placeholder of "https://" in the "image" field.

I have a central folder on a site like dropbox, that has a url structure of https://weburlofnewimage/0, /1, /2 etc. And so I would like to open each file, and change the value of the "image" key to be replaced with "https://weburlofnewimage/ + current file number + '.png'".

The .json currently appears as follows for each json file:


{"image": "https://", "attributes": [{"trait_type": "box color", "value": "blue"}, {"trait_type": "box shape", "value": "square"}]}

but should be

{"image": "https://weburlofnewimage/0", "attributes": [{"trait_type": "box color", "value": "blue"}, {"trait_type": "box shape", "value": "square"}]}

So far I have tried the following, but I believe I am going down a rabbit hole

import json
import os

folderPath = r'/path/FolderWithJson'
fileNumber = 0

for filename in os.listdir(folderPath):
    with open(filename, 'r') as f:
        data = json.load(f)
        data['id'] = str('https://weburlofnewimage/' + str(fileNumber) + '.png')
    os.remove(filename)
    with open(filename, 'w') as f:
        json.dump(data, f, indent=4)

    fileNumber +=1

and have tried:

import os
import json

folderPath = r'/path/FolderWithJson'
fileNumber = 0

for filename in os.listdir(folderPath):
    with open(filename, 'r+') as f:
        data = json.load(f)
        data['image'] = str('https://weburlofnewimage/' + str(fileNumber)
        f.seek(0)
        json.dump(data, f, indent=4)
        f.truncate()     # remove remaining part
        fileNumber +=1

However, in both cases I get the following error:

FileNotFoundError: [Errno 2] No such file or directory: '20.json'

However, the file is indeed in that directory...

Meowsleydale
  • 439
  • 5
  • 15
  • 1
    i bet `filename` doesn't have the absolute path prepended.. wherever you're calling python from is the directory its looking for the file i bet.. try `open(folderPath+'/'+filename, 'r+')` – Jack Oct 02 '21 at 20:40
  • 1
    yep, confirmed that is the case. try to call `os.listdir` on a directory and you'll notice it only retuns the file names (not the full paths) – rv.kvetch Oct 02 '21 at 20:41
  • Thanks for your responses, I tried that (edited above) and got the same error... hmm – Meowsleydale Oct 02 '21 at 21:04

2 Answers2

3

The filename variable only holds the name of the file in the folder you are iterating through, not the full path to that file. You need to specify the full absolute path to your file, or a relative path from where you're running the program.

Using os.path.join will combine the folder relative path you defined and the filename in that folder

for filename in os.listdir(folderPath):
    filePath = os.path.join(folderPath, filename)
    with open(filePath, 'r+') as f:
        # and so on

Also, (as you have found out) assuming that every file in a folder will be a json is unsafe. You should also add a guard to verify it's a .json.

for filename in os.listdir(folderPath):
    if not filename.endswith(".json"): continue
    filePath = os.path.join(folderPath, filename)
    with open(filePath, 'r+') as f:
        # and so on
Andrew-Harelson
  • 1,022
  • 3
  • 11
  • 1
    It looks like you're running this on a mac. I suspect you are parsing hidden `DS_store` files without knowing it like in this question: https://stackoverflow.com/questions/38518023/unicodedecodeerror-utf8-codec-cant-decode-byte-0x80-in-position-3131-invali I've updated my answer to include a guard for non json files. – Andrew-Harelson Oct 02 '21 at 21:43
  • After deleting a .DS file in the folder this is working, except it seems to be opening files randomly. For example, I added some print statements, and first it is opening files 20, then 16, then 6, etc, and so file 20 ends up with image link 0, etc. Any idea how I can make it go sequentially? Thanks again for your help! – Meowsleydale Oct 02 '21 at 21:45
  • I tried your revised code and it beats deleting the DS file every time. However, I notice it is always the same file order, 20, 16, 6, etc. I guess dictionaries are random, but still, Im using python 3.9, and the fact its the same number sequence each time is strange. – Meowsleydale Oct 02 '21 at 21:57
  • 1
    I would either a) read the number from the file name to decide which picture to assign. (more easily updatable/scalable) or b) use the method where you're building the path with a fileNumber counter like in some of the other answers. For this route I would advise against using `os.listdir` at all and instead use `for fileNumber in range(25)` – Andrew-Harelson Oct 02 '21 at 21:58
  • 1
    Also yes the order for `os.listdir` is not random. The documentation defines it as "arbitrary" and a little further research indicates the exact order depends on the file system you use. – Andrew-Harelson Oct 02 '21 at 22:00
  • Thanks a lot for your help! It works great now, except for the out of order thing, but you have given me some ideas to try, so will play around with this some more. But at least the data looks perfect (I actually opened up one randomly and it was the same number image as the file, and thought it was there, but the next one was different, so thanks for helping me get much closer! – Meowsleydale Oct 02 '21 at 22:42
2

You have to specifie a path to json file if it isn't in derictory with python file. Example:

import json
import os

folderPath = r'/path/FolderWithJson'
fileNumber = 0

for filename in os.listdir(folderPath):
    with open(/path/FolderWithJson/json.json, 'r') as f:
        data = json.load(f)
        data['id'] = str('https://weburlofnewimage/' + str(fileNumber) + '.png')
    os.remove(/path/FolderWithJson/json.json)
    with open(/path/FolderWithJson/json.json, 'w') as f:
        json.dump(data, f, indent=4)

    fileNumber +=1
inemyrovsk
  • 42
  • 4
  • Thanks for your response! I posted the code in my edit that I tried based on your description, and got the error: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 3131: invalid start byte – Meowsleydale Oct 02 '21 at 21:19