Scheduler makes a python script run but doesn't produce a csv file

Question

I've written a script in python to scrape some links from a webpage and write them to a csv file. My script does this in the right way when run it from an IDE.

When I run the same using windows task scheduler, I can see that a command prompt pops up and the script runs and prints result there as well but I don't get any csv file when the task is done.

Am I missing something?

What possible change should I bring about to get a csv file when the script is run via .bat file from windows task scheduler?

.bat contains:

@echo off
"C:\Users\WCS\AppData\Local\Programs\Python\Python36-32\python.exe" "C:\Users\WCS\AppData\Local\Programs\Python\Python36-32\examplescript.py"

examplescript.py contains:

import csv
import requests
from urllib.parse import urljoin
from bs4 import BeautifulSoup

url = "https://www.janglo.net/component/option,com_sobi2/"

def get_info(link):
    res = requests.get(url)
    soup = BeautifulSoup(res.text,"lxml")
    for items in soup.select("#sobi2CatListSymbols .sobi2SubcatsListItems a[title]"):
        if items.text=="Tutors":
            ilink = f"{urljoin(url,items.get('href'))}"
    return ilink

def get_links(tlink):
    linklist = []
    res = requests.get(tlink)
    soup = BeautifulSoup(res.text,"lxml")
    for item in soup.select(".sobi2ItemTitle a"):
        linklist.append(urljoin(url,item.get("href")))
    return linklist

if __name__ == '__main__':
    with open("task_scheduler.csv","a",newline="") as infile:
        writer = csv.writer(infile)
        item = get_info(url)
        for nlink in get_links(item):
            print(nlink)
            writer.writerow([nlink])

I've clearly specified the location of the ".bat" file below:

"C:\Users\WCS\AppData\Local\Programs\Python\Python36-32\test_schedule.bat"

You don't need a single-line batch file to run the Python script as a task. I don't know why you think you need it. — Eryk Sun, Dec 02 '18 at 14:47
That is because I could not find any way to save the csv file in the python directory without executing `.bat` file @eryksun. — SIM, Dec 03 '18 at 11:02
Put the fully-qualified path of "python.exe" in the "program" field; the fully-qualified and quoted path of the script in the "arguments" field; and the fully-qualified working directory in the "start in" field. That said, you shouldn't save data files in the Python directory or in application directories, for various reasons, including the fact that in general your program may not have permission to create files there. For user-visible data, use well-known user data folders such as "Documents". — Eryk Sun, Dec 03 '18 at 14:38

Martijn Pieters · Accepted Answer · 2018-12-02T15:33:44.580

3

Your code writes to the current working directory, because it opens a filename without a path. The location of the .bat file does not determine the current working directory of code being run.

You didn't specify a working directory; I'd expect that to be set with the Start in field in the scheduler, so the file is being created in the default location, %Windir%\System32\.

Either set a directory in the Start in field for the CSV file to be created in, have your batch script use cd to move the current working directory to the right location, or use an absolute path when opening the file.

For example, if your batch script first runs cd /d %~dp0 then the working directory is changed to the directory of the batch script, and relative paths in Python are resolved against that location.

edited Dec 02 '18 at 15:33

answered Dec 02 '18 at 13:57

Martijn Pieters

1,048,767
296
4,058
3,343

Right you are @sir Martijn Pieters!! I found a csv file containing those required fields in that default directory (system32). However, when I specify this `C:\Users\WCS\Desktop` address right next to `Start in (optional):` inputbox (visible in the image above), I still get the csv file in the default directory. Did I follow your instruction properly? – SIM Dec 02 '18 at 14:35
@asmitu: I don't know. I don't use Windows myself, and my research shows that that option *should* set the current working directory for the command you are running. However, in case that the batch script CWD doesn't carry forward, I added another option: `cd /d %~dp0`. – Martijn Pieters Dec 02 '18 at 15:35

Scheduler makes a python script run but doesn't produce a csv file

1 Answers1