0

I have a Python script that does some web-scraping, then opens and dumps the parsed data into a JSON file in the same directory. Everything works, when the script is run manually through the CLI, but the data does not get written to the JSON file, when run from the batch file run by a task scheduler.

I have managed to show that all the data exists within the Python script, when run through the batch file. Somehow only part of the function that deals with the JSON file is not run.

Python script:

# Packages used:
import requests
from bs4 import BeautifulSoup
import smtplib
import time
from win10toast import ToastNotifier
import json

# Web Scraping...

my_json = {}

def function1():
    # Web scraping for data...

    json_function(data)

# Below is the function that is not functioning
def json_function(data):
    my_json[time.strftime("%Y-%m-%d %H:%M")] = f"{data}"
    with open ('json_file.json') as my_dict:
        info = json.load(my_dict)
    info.update(my_json)

    with open('json_file.json','w') as my_dict:
        json.dump(info,my_dict)

# A few other functions that work regardless...

# Call function
function1()

Batch file:

"C:\Users\...pythonw.exe" "C:\Users...script.pyw"

JSON file:

{"Key":"Value"}

Every file is in the same directory.

When run from the CLI, expected result occurs - key-value are appended to the JSON file. When run automatically (through batch and task scheduler), no visible errors, and all of the script, save for the json_function, run as expected.

Mofi
  • 46,139
  • 17
  • 80
  • 143
Sky020
  • 47
  • 1
  • 7
  • try to pass the fullpath of the json file when you open it – PRMoureu Jul 24 '19 at 11:17
  • @PRMoureu Thank you. That seems to have solved it. Could you explain why? – Sky020 Jul 24 '19 at 11:27
  • it's about the current working directory, use `import os;print(os.getcwd())` to see why you cannot reach the file if you launch the script from another directory – PRMoureu Jul 24 '19 at 11:46
  • 1
    See [What must be taken into account on executing a batch file as scheduled task?](https://stackoverflow.com/a/41821620/3074564) You have forgotten to configure in properties of scheduled task the __Start in__ (current directory) folder and for that reason the current directory is set to `%SystemRoot%\System32` on starting the scheduled task. BTW: There is no batch file needed at all. Just configure in scheduled task to run `"C:\Users\...pythonw.exe"` with the argument `"C:\Users...script.pyw"`, except the batch file contains more than this line which cannot be done with Python script itself. – Mofi Jul 24 '19 at 11:59
  • See also [How do I get the path of the Python script I am running in?](https://stackoverflow.com/questions/595305/) and [In Python, how do I get the path and name of the file that is currently executing?](https://stackoverflow.com/questions/50499/) A good coded program or script which accesses files in program/script folder determine first the folder path of running program/script, concatenate this folder path with the names of the files to open/save in program/script folder and use the dynamically created full qualified file names instead of just file name without path. – Mofi Jul 24 '19 at 12:04

1 Answers1

0

Thank you to @PRMoureu for the answer, and @Mofi for a detailed explanation.

Answer is to ensure all files referenced have their full path referenced:

    def json_function(data):
    my_json[time.strftime("%Y-%m-%d %H:%M")] = f"{data}"
    with open ('C:/.../json_file.json') as my_dict:
        info = json.load(my_dict)
    info.update(my_json)

    with open('C:/.../json_file.json','w') as my_dict:
        json.dump(info,my_dict)

Or, direct the Task Scheduler to the working directory to avoid the batch being run in the default, root directory.

Sky020
  • 47
  • 1
  • 7