Trying to call a Python 3 script from a batch file when Python 2 is the default

Question

I have a batch file that calls mostly Python 2 scripts, however at the very beginning it needs to call a Python 3 script.

My Python 2 path (default, this is what is returned when I call python -V):

C:\Python27\python.exe

My Python 3 path:

C:\Users\Isaac\Anaconda3\python.exe

(The weird file structure is because I use an IDE that came with a pre-built python installation)

This is the batch file (I don't know if I actually need the timeout command or not):

for /f "tokens=2 delims=:." %%x in ('chcp') do set cp=%%x
chcp 1252>nul
start C:\Users\Isaac\Anaconda3\python.exe opennorth_downloader.py %1
timeout /t 10 /nobreak
gen_outline.py --collection objects F:\electoral_map\%1\candidates\candidates_Bloc_Québécois.json
json2csv.py F:\electoral_map\%1\candidates\candidates_Bloc_Québécois.json F:\electoral_map\%1\candidates\candidates_Bloc_Québécois.outline.json -o F:\electoral_map\%1\candidates\Bloc_Québécois.csv
del F:\electoral_map\%1\candidates\candidates_Bloc_Québécois.json
del F:\electoral_map\%1\candidates\candidates_Bloc_Québécois.outline.json
gen_outline.py --collection objects F:\electoral_map\%1\candidates\candidates_Christian_Heritage.json
json2csv.py F:\electoral_map\%1\candidates\candidates_Christian_Heritage.json F:\electoral_map\%1\candidates\candidates_Christian_Heritage.outline.json -o F:\electoral_map\%1\candidates\Christian_Heritage.csv
del F:\electoral_map\%1\candidates\candidates_Christian_Heritage.json
del F:\electoral_map\%1\candidates\candidates_Christian_Heritage.outline.json
gen_outline.py --collection objects F:\electoral_map\%1\candidates\candidates_Conservative.json
json2csv.py F:\electoral_map\%1\candidates\candidates_Conservative.json F:\electoral_map\%1\candidates\candidates_Conservative.outline.json -o F:\electoral_map\%1\candidates\Conservative.csv
del F:\electoral_map\%1\candidates\candidates_Conservative.json
del F:\electoral_map\%1\candidates\candidates_Conservative.outline.json
gen_outline.py --collection objects F:\electoral_map\%1\candidates\candidates_Forces_et_Démocratie.json
json2csv.py F:\electoral_map\%1\candidates\candidates_Forces_et_Démocratie.json F:\electoral_map\%1\candidates\candidates_Forces_et_Démocratie.outline.json -o F:\electoral_map\%1\candidates\Forces_et_Démocratie.csv
del F:\electoral_map\%1\candidates\candidates_Forces_et_Démocratie.json
del F:\electoral_map\%1\candidates\candidates_Forces_et_Démocratie.outline.json
gen_outline.py --collection objects F:\electoral_map\%1\candidates\candidates_Green_Party.json
json2csv.py F:\electoral_map\%1\candidates\candidates_Green_Party.json F:\electoral_map\%1\candidates\candidates_Green_Party.outline.json -o F:\electoral_map\%1\candidates\Green_Party.csv
del F:\electoral_map\%1\candidates\candidates_Green_Party.json
del F:\electoral_map\%1\candidates\candidates_Green_Party.outline.json
gen_outline.py --collection objects F:\electoral_map\%1\candidates\candidates_Liberal.json
json2csv.py F:\electoral_map\%1\candidates\candidates_Liberal.json F:\electoral_map\%1\candidates\candidates_Liberal.outline.json  -o F:\electoral_map\%1\candidates\Liberal.csv
del F:\electoral_map\%1\candidates\candidates_Liberal.json
del F:\electoral_map\%1\candidates\candidates_Liberal.outline.json
gen_outline.py --collection objects F:\electoral_map\%1\candidates\candidates_Libertarian.json
json2csv.py F:\electoral_map\%1\candidates\candidates_Libertarian.json F:\electoral_map\%1\candidates\candidates_Libertarian.outline.json -o F:\electoral_map\%1\candidates\Libertarian.csv
del F:\electoral_map\%1\candidates\candidates_Libertarian.json
del F:\electoral_map\%1\candidates\candidates_Libertarian.outline.json
gen_outline.py --collection objects F:\electoral_map\%1\candidates\candidates_NDP.json
json2csv.py F:\electoral_map\%1\candidates\candidates_NDP.json F:\electoral_map\%1\candidates\candidates_NDP.outline.json -o F:\electoral_map\%1\candidates\NDP.csv
del F:\electoral_map\%1\candidates\candidates_NDP.json
del F:\electoral_map\%1\candidates\candidates_NDP.outline.json
chcp %cp%>nul
pause

I know that everything works except for this line:

start C:\Users\Isaac\Anaconda3\python.exe opennorth_downloader.py %1

(The batch file worked when it was run without it with what it does set up in advance)

This is the script that it references:

from sys import argv
import urllib

party_name_list = ["Conservative", "Liberal", "NDP", "Green Party", "Bloc Québécois", "Forces et Démocratie", "Libertarian", "Christian Heritage"]

for party_name in party_name_list:
    with urllib.request.urlopen(r"https://represent.opennorth.ca/candidates/house-of-commons/?limit=1000&party_name={}".format(urllib.parse.quote_plus(party_name))) as url:
        with open(r"F:\electoral_map\{0}\candidates\candidates_{1}.json".format(argv[1], party_name.replace(' ', '_')), "wb+") as f:
            f.write(url.read())
    print("finished {0}".format(party_name))
print("all done")

The batch file is called with a single argument: "20150915" which represents a date and points everything to a single folder that everything is contained within.

Can anybody see a reason why that problem line would fail? It looks like what it is supposed to be according to this

I guess I don't know for sure, the cmd window that I call from batch file from launches a new window that instantly terminates, but all the rest of the commands fail due to lack of pre-requisite files that are supposed to be generated by that "problem line". If I set up the folder with the outputs of opennorth_downloader after I run it in my IDE, and then exclude the batch file without that problem line, the batch file executes without incident. — wfgeo, Sep 16 '15 at 02:14
No, it doesn't. I will try that when I get home, thanks, wOxxOm — wfgeo, Sep 16 '15 at 16:04
try to replace `%1` by `%~1` in `start C:\Users\Isaac\Anaconda3\python.exe opennorth_downloader.py %1` or try `set "myvar=%~1" & start C:\Users\Isaac\Anaconda3\python.exe opennorth_downloader.py %myvar%` — Paul, Sep 16 '15 at 17:38
wOxxOm and Paul, those didn't work :/ I changed my PATH to point to my Python 3 installation instead of Python 2 but I get the error AttributeError: 'module' objects has no attribute 'request' on line 9... But it works fine when I run it from my IDE — wfgeo, Sep 17 '15 at 00:41
@1saac, probably related: http://stackoverflow.com/questions/22222473/shebang-doesnt-work-with-python3 — wOxxOm, Sep 17 '15 at 12:33

score 0 · Answer 1 · answered Sep 17 '15 at 00:57

0

I was finally able to solve it after a LOT of digging and mucking around. For some unknown reason, the .bat does not recognize the "request" module of urllib unless it is explicitly imported, like so:

from sys import argv
from urllib import request
from urllib import parse

party_name_list = ["Conservative", "Liberal", "NDP", "Green Party", "Bloc Québécois", "Forces et Démocratie", "Libertarian", "Christian Heritage"]

for party_name in party_name_list:
    with request.urlopen(r"https://represent.opennorth.ca/candidates/house-of-commons/?limit=1000&party_name={}".format(parse.quote_plus(party_name))) as url:
        with open(r"F:\electoral_map\{0}\candidates\candidates_{1}.json".format(argv[1], party_name.replace(' ', '_')), "wb+") as f:
            f.write(url.read())
    print("finished {0}".format(party_name))
print("all done")

Then it works with no issue using the same batch file as above. I have no idea why but at this point I'm just happy it's working :)

answered Sep 17 '15 at 00:57

wfgeo

2,716
4
30
51

in general, `import a` does not also import `a.b`. If you need `a.b`; you must `import a.b`. – jfs Sep 17 '15 at 01:06
unrelated: you could use `shutil.copyfileobj(url, f)`, to avoid loading the whole content into memory first or just use `urllib.request.urlretrieve('https://...', r'F:\...json')`. You could also [make the http requests concurrently](http://stackoverflow.com/a/31795242/4279) – jfs Sep 17 '15 at 01:12
J.F. Sebastian: I'm aware of that, but my original script had the full name in a.b.c form, for some reason the batch file just didn't like having to call a.b.c even when all of a was imported. The script actually worked in my IDE, just failed when called in cmd – wfgeo Sep 17 '15 at 01:14
Try from the command-line: `py -3 -c "import urllib; urllib.request"` -- it should fail. If it doesn't fail in your IDE; it is broken (it doesn't isolate the execution of your script sufficiently). `py -3 -c "import urllib.request; urllib.request"` should not fail. – jfs Sep 17 '15 at 01:16
I'm afraid I'm not following you. If I import a, and then call a.b.c and it works, how could the IDE be broken? – wfgeo Sep 17 '15 at 01:25
1

because it means that IDE does things behind your back. In your case, it makes your incorrect program run without an error (who knows what other side-effects it introduces -- it is not safe to import arbitrary modules left and right -- any code can be run at import time -- prey that you don't have a module with `shutil.rmtree('/')` lying around). You must `import urllib.request` if you want to use `urllib.request` module. – jfs Sep 17 '15 at 01:48

Trying to call a Python 3 script from a batch file when Python 2 is the default

1 Answers1